Config-driven local/Databricks/Trino switching #20

Closed
opened 2026-03-12 17:40:14 +00:00 by kert · 0 comments
Owner

stack run --target duckdb|databricks|trino

All three targets must:

  • Use identical pipeline code (same Expr/fn)
  • Differ only in the loader function
  • Be selectable via CLI flag or stack.toml [default] section
  • Produce identical output schemas (validated by SchemaError)

Add [default] section to stack.toml:

[default]
target = "duckdb"
stack run <pipeline> --target duckdb|databricks|trino All three targets must: - Use identical pipeline code (same Expr/fn) - Differ only in the loader function - Be selectable via CLI flag or stack.toml [default] section - Produce identical output schemas (validated by SchemaError) Add [default] section to stack.toml: ```toml [default] target = "duckdb" ```
kert added this to the P5: Databricks — Jobs, Unity Catalog, parity milestone 2026-03-12 17:40:14 +00:00
kert added the configpipelinedatabricks labels 2026-03-12 17:40:14 +00:00
kert closed this issue 2026-03-12 21:16:19 +00:00
Sign in to join this conversation.