kert 1baf98eb40 implement data ingestion: CCLF loader, BCDA loader, seed loader, unified staging ...

- CCLF: fixed-width parser using IP-derived field positions (cclf_layout.py),
  ZIP extraction, file discovery by CMS naming convention, DuckDB loading,
  optional pipeline execution to produce input_layer tables
- BCDA: flatten ndjson → Parquet, load into bcda schema in DuckDB
- Seeds: auto-discover CSV/Excel/Parquet files, load into reference_data schema
- Unified staging: orchestrate all three loaders with optional Iceberg promotion
- CLI: stack load cclf/bcda/seed with full options (--path, --database, etc.)
- 57 tests covering parsers, loaders, CLI commands, and staging pipeline

fixes #5 fixes #6 fixes #7 fixes #8

2026-03-12 15:52:08 -04:00

2.1 KiB

Raw Blame History

View Raw