P22: DAB Generator — pluggable databricks.yml from pipeline registry
Generate valid Databricks Asset Bundle (databricks.yml) config from the pipeline registry, table catalog, and stack.toml settings.
Closeout
All 10 issues closed. Pipeline model enriched with dependency and cluster metadata. [databricks] config section added to stack.toml for bundle generation parameters. gen_dab.py emitter produces databricks.yml from the pipeline registry and config. Unity Catalog schema and volume resources generated from the table catalog. Pluggable cluster profiles support different compute tiers (dev, staging, prod). Artifact configuration generated for Python wheel builds. Permissions and run_as derived from stack.toml. Freshness check in CI via gen_config.py --check ensures databricks.yml stays in sync with the registry. Plugin architecture supports custom DAB resource types. Comprehensive test suite covers round-trip generation, schema validation, drift detection, and extensibility.
Issues resolved
- #170 — Pipeline model enriched with dependency/cluster metadata
- #171 — [databricks] config section in stack.toml
- #172 — gen_dab.py emitter: registry + config → databricks.yml
- #173 — Unity Catalog schema/volume resources from table catalog
- #174 — Pluggable cluster profiles for compute tiers
- #175 — Artifact config for Python wheel build
- #176 — Permissions and run_as from stack.toml
- #177 — Freshness check in CI (gen_config.py --check)
- #178 — Plugin architecture for custom DAB resource types
- #179 — Round-trip, validation, drift, extensibility tests
Residual
None. Superseded by P28 (DAB Modernization) which replaced SQL transpilation with Python wheel tasks.
