kert
|
f2e9dabadb
|
add DAB generator: databricks.yml from pipeline registry + stack.toml
Infra CI / notebooks (push) Successful in 7s
Infra CI / zotero (push) Successful in 8s
CI / lint-test (push) Successful in 1m23s
Infra CI / docs (push) Successful in 10s
Infra CI / api (push) Successful in 26s
Infra CI / mc (push) Successful in 6s
Infra CI / notebooks (pull_request) Successful in 13s
Infra CI / zotero (pull_request) Successful in 7s
Infra CI / docs (pull_request) Successful in 6s
Infra CI / api (pull_request) Successful in 6s
Infra CI / mc (pull_request) Successful in 7s
CI / lint-test (pull_request) Successful in 1m17s
closes #170, closes #171, closes #172, closes #173, closes #174,
closes #175, closes #176, closes #177, closes #178, closes #179
Pipeline metadata (#170):
- Add upstream and cluster_profile to Pipeline model
- Populate upstream for all 13 pipeline modules
Config (#171):
- Add [databricks] section to stack.toml with clusters, targets
Core emitter (#172):
- backends/databricks.py generates databricks.yml from registry
- All 13 tasks with correct depends_on, no hardcoding
- Wired into gen_config.py (generate + --check)
Resources (#173, #174):
- 30 Unity Catalog schemas from aco.lake.catalog.Catalog
- Staging volume for data upload
- Cluster profiles: only referenced profiles emitted
Artifacts (#175): wheel build via uv build
Permissions (#176): production targets support run_as
CI gate (#177): gen_config.py --check covers databricks.yml
Plugin architecture (#178): DabPlugin protocol, 3 built-in plugins
Tests (#179): 23 tests covering round-trip, completeness, deps, idempotency
|
2026-03-24 01:57:14 -04:00 |
|