kert
0 Followers · 0 Following
  • Joined on 2026-02-04
Repositories Projects Packages Public Activity Starred Repositories
kert commented on issue homelab/stack#222 2026-03-25 03:39:34 +00:00
dab: add CLI --pipeline/--catalog flags for Databricks wheel execution

Complete.

  • Added --catalog option to stack run (implies target=spark)
  • Added _SparkContext class with make_databricks_loader integration
  • stack run readmissions --catalog aco_dev…
kert closed issue homelab/stack#221 2026-03-25 03:34:43 +00:00
dab: regenerate bundle/sql/ to match current pipeline code
kert commented on issue homelab/stack#221 2026-03-25 03:34:43 +00:00
dab: regenerate bundle/sql/ to match current pipeline code

Complete.

Regenerated all 208 SQL files via --dab-sql. Result: zero diff — SQL was already in sync with current pipeline code.

Note: cms_quality_measures/_error.txt is a known transpile…

kert closed issue homelab/stack#220 2026-03-25 03:34:43 +00:00
dab: remove unused large cluster config from stack.toml
kert commented on issue homelab/stack#220 2026-03-25 03:34:43 +00:00
dab: remove unused large cluster config from stack.toml

Complete.

  • Removed [databricks.clusters.large] from stack.toml (i3.2xlarge, 4 workers — unused, no pipeline references it)
  • Regenerated databricks.yml — only default_cluster remains

✅…

kert closed issue homelab/stack#219 2026-03-25 03:34:43 +00:00
dab: filter Unity Catalog schemas to pipeline-produced only
kert commented on issue homelab/stack#219 2026-03-25 03:34:43 +00:00
dab: filter Unity Catalog schemas to pipeline-produced only

Complete.

  • Added Catalog.pipeline_schemas() — derives schemas from pipeline registry output names
  • Updated _get_catalog_schemas() in DAB generator to use pipeline_schemas()
  • Result: 13…
kert commented on issue homelab/stack#221 2026-03-25 03:28:56 +00:00
dab: regenerate bundle/sql/ to match current pipeline code

Starting SQL regeneration. This takes ~4 minutes (transpiles all 228 expressions).

kert commented on issue homelab/stack#219 2026-03-25 03:23:55 +00:00
dab: filter Unity Catalog schemas to pipeline-produced only

Starting. Will add pipeline_schemas() to catalog.py and use it in the DAB generator.

kert commented on issue homelab/stack#220 2026-03-25 03:23:55 +00:00
dab: remove unused large cluster config from stack.toml

Starting in parallel with #219.

kert opened issue homelab/stack#226 2026-03-25 03:23:18 +00:00
dab: final validation — databricks bundle validate, tests, cleanup
kert opened issue homelab/stack#224 2026-03-25 03:23:18 +00:00
dab: add databricks-bundles package and Python resource loader
kert opened issue homelab/stack#223 2026-03-25 03:23:18 +00:00
dab: replace sql_task with python_wheel_task in JobsPlugin
kert opened issue homelab/stack#222 2026-03-25 03:23:18 +00:00
dab: add CLI --pipeline/--catalog flags for Databricks wheel execution
kert opened issue homelab/stack#221 2026-03-25 03:23:18 +00:00
dab: regenerate bundle/sql/ to match current pipeline code
kert opened issue homelab/stack#225 2026-03-25 03:23:18 +00:00
dab: retain SQL transpilation as validation/audit tool only
kert opened issue homelab/stack#219 2026-03-25 03:23:17 +00:00
dab: filter Unity Catalog schemas to pipeline-produced only
kert opened issue homelab/stack#220 2026-03-25 03:23:17 +00:00
dab: remove unused large cluster config from stack.toml
kert pushed to main at homelab/stack 2026-03-25 02:50:26 +00:00
6d291334ca fix: mirror sync also skips local project name
kert pushed to main at homelab/stack 2026-03-25 02:38:12 +00:00
1e709afc31 fix: pkg inventory parser — filter self-refs, parse optional-dependencies