85ce5e719d
Some checks failed
CI / skinny-install (aco) (push) Successful in 45s
CI / skinny-install (api) (push) Successful in 29s
CI / skinny-install (bcda) (push) Successful in 25s
CI / skinny-install (bib) (push) Successful in 23s
CI / skinny-install (bls) (push) Successful in 20s
CI / skinny-install (ccw) (push) Successful in 36s
CI / skinny-install (cli) (push) Successful in 27s
CI / skinny-install (cms) (push) Successful in 24s
CI / skinny-install (conf) (push) Successful in 27s
CI / skinny-install (pfs) (push) Successful in 25s
CI / skinny-install (rex) (push) Successful in 25s
CI / lint-test (push) Successful in 6m2s
Infra CI / notebooks (push) Successful in 7s
Infra CI / zotero (push) Failing after 6s
Infra CI / docs (push) Successful in 33s
Infra CI / api (push) Successful in 6s
Infra CI / mc (push) Successful in 7s
Deploy / build-scan-report (push) Has been cancelled
- Fix all 72 ruff lint errors (unused imports, unused variables, E402) - Format all 14 unformatted dev/scripts files - Move generated artifacts to assets/ (dag.html, pfs.html) - Remove duplicate root coverage.svg (already in assets/icons/) - Update .dockerignore for infra/ tree layout - Update .gitignore: add .env.bak, mirrors/, htmlcov/ - Fix stale path refs in coverage_badge.py, woodpecker backend, test_network_isolation.sh, docs custom.css - Add .gitkeep to empty dirs (infra/polaris, cloud/*/terraform) - Delete 12 stale local branches, 10 stale remote branches
GCP Deployment
Service Mapping
| Component | GCP Service | Replaces |
|---|---|---|
| Object Storage | Cloud Storage (GCS) | RustFS |
| Database | Cloud SQL (PostgreSQL) | PostgreSQL container |
| Iceberg Catalog | BigQuery / Dataplex | Nessie / Polaris |
| Query Engine | BigQuery | Trino |
| Compute | Cloud Run / GKE | Docker Compose |
| Ingress | Cloud Load Balancing + DNS | Traefik + CoreDNS |
| CI/CD | Cloud Build / GitHub Actions | Gitea Actions |
| Observability | Cloud Monitoring + Trace | Grafana stack |
Context
[context.gcp]
db_backend = "iceberg"
storage_backend = "gcs"
catalog = "bigquery"
compute = "cloud-run"
Environment Variables
export STACK_CONTEXT=gcp
export GOOGLE_CLOUD_PROJECT=your-project-id
export GCS_BUCKET=your-lakehouse-bucket
# Workload Identity handles auth on Cloud Run
Required IAM Roles
roles/storage.objectAdminon lakehouse bucketroles/bigquery.dataEditorfor catalogroles/cloudsql.clientfor PostgreSQL
Terraform
See terraform/ for resource definitions (placeholder).