85ce5e719d
Some checks failed
CI / skinny-install (aco) (push) Successful in 45s
CI / skinny-install (api) (push) Successful in 29s
CI / skinny-install (bcda) (push) Successful in 25s
CI / skinny-install (bib) (push) Successful in 23s
CI / skinny-install (bls) (push) Successful in 20s
CI / skinny-install (ccw) (push) Successful in 36s
CI / skinny-install (cli) (push) Successful in 27s
CI / skinny-install (cms) (push) Successful in 24s
CI / skinny-install (conf) (push) Successful in 27s
CI / skinny-install (pfs) (push) Successful in 25s
CI / skinny-install (rex) (push) Successful in 25s
CI / lint-test (push) Successful in 6m2s
Infra CI / notebooks (push) Successful in 7s
Infra CI / zotero (push) Failing after 6s
Infra CI / docs (push) Successful in 33s
Infra CI / api (push) Successful in 6s
Infra CI / mc (push) Successful in 7s
Deploy / build-scan-report (push) Has been cancelled
- Fix all 72 ruff lint errors (unused imports, unused variables, E402) - Format all 14 unformatted dev/scripts files - Move generated artifacts to assets/ (dag.html, pfs.html) - Remove duplicate root coverage.svg (already in assets/icons/) - Update .dockerignore for infra/ tree layout - Update .gitignore: add .env.bak, mirrors/, htmlcov/ - Fix stale path refs in coverage_badge.py, woodpecker backend, test_network_isolation.sh, docs custom.css - Add .gitkeep to empty dirs (infra/polaris, cloud/*/terraform) - Delete 12 stale local branches, 10 stale remote branches
AWS Deployment
Service Mapping
| Component | AWS Service | Replaces |
|---|---|---|
| Object Storage | S3 | RustFS |
| Database | RDS PostgreSQL | PostgreSQL container |
| Iceberg Catalog | AWS Glue / Lake Formation | Nessie / Polaris |
| Query Engine | Athena / EMR | Trino |
| Compute | ECS Fargate / EKS | Docker Compose |
| Ingress | ALB + Route 53 | Traefik + CoreDNS |
| CI/CD | GitHub Actions | Gitea Actions |
| Observability | CloudWatch + X-Ray | Grafana stack |
Context
[context.aws]
db_backend = "iceberg"
storage_backend = "s3"
catalog = "glue"
compute = "ecs"
Environment Variables
export STACK_CONTEXT=aws
export AWS_DEFAULT_REGION=us-east-1
export AWS_S3_BUCKET=your-lakehouse-bucket
# IAM role handles auth — no access keys needed on ECS
Required IAM Permissions
s3:GetObject,s3:PutObject,s3:ListBucketon lakehouse bucketglue:GetTable,glue:CreateTable,glue:UpdateTablefor catalogrds-db:connectfor PostgreSQL
Terraform
See terraform/ for resource definitions (placeholder).