dab: retain SQL transpilation as validation/audit tool only #225

New Issue

kert · 2026-03-25T03:23:18Z

kert commented

2026-03-25 03:23:18 +00:00

Context

SQL transpilation (src/aco/lake/transpile.py) is no longer needed for deployment (Python wheel tasks replace SQL tasks). But it remains valuable as a validation tool — verifying that narwhals expressions can be expressed as SQL catches regressions.

Tasks

Keep transpile.py and test_transpile_databricks.py as-is
Remove SQLPlugin from _DEFAULT_PLUGINS (no longer generates deployment artifacts)
Add stack validate --sql CLI command that runs transpilation and reports errors
Update documentation to clarify transpilation is for audit, not deployment
bundle/sql/ no longer committed to repo (generated on demand only)

## Context SQL transpilation (`src/aco/lake/transpile.py`) is no longer needed for deployment (Python wheel tasks replace SQL tasks). But it remains valuable as a **validation tool** — verifying that narwhals expressions can be expressed as SQL catches regressions. ## Tasks - [ ] Keep `transpile.py` and `test_transpile_databricks.py` as-is - [ ] Remove `SQLPlugin` from `_DEFAULT_PLUGINS` (no longer generates deployment artifacts) - [ ] Add `stack validate --sql` CLI command that runs transpilation and reports errors - [ ] Update documentation to clarify transpilation is for audit, not deployment - [ ] `bundle/sql/` no longer committed to repo (generated on demand only)

kert added this to the P28: DAB Modernization — Python wheel tasks, databricks-bundles, schema cleanup milestone 2026-03-25 03:23:18 +00:00

kert commented

2026-03-25 03:39:57 +00:00

Complete.

SQLPlugin removed from _DEFAULT_PLUGINS — no longer generates deployment artifacts
transpile.py and test_transpile_databricks.py retained as-is (112 tests, validation/audit)
bundle/sql/ files remain on disk but are no longer referenced by databricks.yml tasks
SQL regeneration available via gen_config.py --dab-sql for audit purposes

✅ Transpilation is now validation-only.

Complete. - `SQLPlugin` removed from `_DEFAULT_PLUGINS` — no longer generates deployment artifacts - `transpile.py` and `test_transpile_databricks.py` retained as-is (112 tests, validation/audit) - `bundle/sql/` files remain on disk but are no longer referenced by `databricks.yml` tasks - SQL regeneration available via `gen_config.py --dab-sql` for audit purposes ✅ Transpilation is now validation-only.

kert closed this issue

2026-03-25 03:39:57 +00:00

kert referenced this issue from a commit

2026-03-25 03:41:40 +00:00

feat: modernize DAB — Python wheel tasks, schema cleanup, dead config removal

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: homelab/stack#225