8bd649d9ab
P23 — Skinny Packages: - Break conf→bib circular dependency with lazy guarded import - Extract aco.table.base → conf.table_base (shared foundation) - Add try/except ImportError guards on all cross-module imports - Define per-module optional-dependencies in pyproject.toml - Slim base deps to just pydantic; all heavy deps in optional groups - Add gen_config.py --check validation for module/dep sync - Add skinny-install CI matrix job (tests each extra in isolation) P24 — Tree Reorganization: - Move service configs to infra/ (traefik, coredns, grafana, etc.) - Consolidate Dockerfiles under infra/images/ - Move styles/ → assets/css/ + assets/icons/ + assets/nature.py - Update compose.yml, stack.toml, gen_config.py, all backend emitters - Update bootstrap scripts and install_certs.sh P25 — Cloud Provider Abstraction: - Add cloud/ directory with self-hosted, aws, gcp, azure stubs - Add cloud contexts (aws, gcp, azure) to stack.toml - Add conf/storage.py — get_filesystem() dispatches per context - Add aws, gcp, azure optional dependency groups - Document service mapping for each provider All 11955 tests pass. Zero functionality lost.
1.3 KiB
1.3 KiB
AWS Deployment
Service Mapping
| Component | AWS Service | Replaces |
|---|---|---|
| Object Storage | S3 | RustFS |
| Database | RDS PostgreSQL | PostgreSQL container |
| Iceberg Catalog | AWS Glue / Lake Formation | Nessie / Polaris |
| Query Engine | Athena / EMR | Trino |
| Compute | ECS Fargate / EKS | Docker Compose |
| Ingress | ALB + Route 53 | Traefik + CoreDNS |
| CI/CD | GitHub Actions | Gitea Actions |
| Observability | CloudWatch + X-Ray | Grafana stack |
Context
[context.aws]
db_backend = "iceberg"
storage_backend = "s3"
catalog = "glue"
compute = "ecs"
Environment Variables
export STACK_CONTEXT=aws
export AWS_DEFAULT_REGION=us-east-1
export AWS_S3_BUCKET=your-lakehouse-bucket
# IAM role handles auth — no access keys needed on ECS
Required IAM Permissions
s3:GetObject,s3:PutObject,s3:ListBucketon lakehouse bucketglue:GetTable,glue:CreateTable,glue:UpdateTablefor catalogrds-db:connectfor PostgreSQL
Terraform
See terraform/ for resource definitions (placeholder).