Stack

Healthcare analytics platform on self-hosted infrastructure. Replaces dbt SQL models with narwhals DataFrame-agnostic expression functions, backed by DuckDB locally and Iceberg/Trino in the lakehouse. 22 services, 13 data pipelines, 11,931 tests at 99% coverage.

Quick start

# 1. Generate root key and seed .env
ROOT_KEY=$(openssl rand -hex 32)
cat > .env <<EOF
DOMAIN=homelab.fhirworx.io
HOST_IP=192.168.1.192
WOODPECKER_ADMIN=kert
EOF

# 2. Start core services and bootstrap credentials
docker compose up -d postgres rustfs traefik
ROOT_KEY=$ROOT_KEY uv run python -m api.auth bootstrap $(git rev-parse HEAD)

# 3. Start everything
docker compose up -d

# 4. Verify
curl -s http://localhost:8000/health | python3 -m json.tool

Services

All services route through Traefik at *.homelab.fhirworx.io.

Service	URL	Purpose
API	api (internal)	FastAPI — health, pipelines, bib endpoints
Notebooks	notebooks.homelab.fhirworx.io	Marimo notebooks (GPU-accelerated)
Docs	docs.homelab.fhirworx.io	Docusaurus — API reference + CMS bibliography
Gitea	gitea.homelab.fhirworx.io	Git server, container registry, LFS
Woodpecker CI	ci.homelab.fhirworx.io	CI/CD pipelines
Zotero	zotero.homelab.fhirworx.io	Reference manager (KasmVNC desktop)
Dashboard	homelab.fhirworx.io	Service launcher
RustFS	s3.homelab.fhirworx.io	S3-compatible object storage
Nessie	nessie.homelab.fhirworx.io	Git-like Iceberg catalog
Polaris	polaris.homelab.fhirworx.io	Iceberg catalog with governance
Trino	trino.homelab.fhirworx.io	Distributed SQL engine
Grafana	grafana.homelab.fhirworx.io	Dashboards
Prometheus	prometheus.homelab.fhirworx.io	Metrics
Jaeger	jaeger.homelab.fhirworx.io	Distributed tracing
Loki	loki.homelab.fhirworx.io	Log aggregation

Analytics platform

Architecture

  CCLF/BCDA/CMS data
        │
        ▼
  src/aco/express/     narwhals expressions (DataFrame-agnostic)
        │
        ▼
  src/aco/pipe/        pipeline runner with schema validation
        │
        ▼
  DuckDB (local)  ──or──  Iceberg + Trino (lakehouse)
        │
        ▼
  src/api/             FastAPI (health, runs, bib)
        │
        ▼
  notebooks/           Marimo interactive analysis

Pipelines

13 registered pipelines with 189 steps, validated by run_pipeline(steps, load) which enforces output schemas via pandera-style column checks.

Pipeline	Domain
input_layer	Raw CCLF/BCDA ingestion
core	Encounters, patients, providers
claims_preprocessing	Claim matching, encounter IDs
readmissions	30-day readmission rates
pharmacy	Part D drug utilization
ahrq_measures	AHRQ quality indicators
cms_quality_measures	CMS ACO quality metrics
hcc_suspecting	HCC risk adjustment
provider_attribution	Provider-patient assignment
quality_measures	Composite quality scores
data_quality	Validation and completeness
cclf	CCLF file parsing
main	Full pipeline orchestration

Expression layer

src/aco/express/ contains narwhals functions that work across polars, pandas, and cuDF. Each module mirrors a pipeline and exports pure transformation functions.

Configuration

stack.toml is the single config file. src/conf/ provides:

• cfg — attribute access (cfg.db.aco)
• path("db.aco") — absolute path resolution relative to repo root
• context() — environment switching (local/lake/databricks/trino)
• secret("api.secret", "STACK_API_SECRET") — env var with config fallback

Override context at runtime: STACK_CONTEXT=lake.

Bibliography

src/bib/ manages a Zotero-backed citation store in data/bib.sqlite. Column-level provenance via Tag.col(ref, desc). The docs site exports library.json for a searchable bibliography browser at /library.

CI/CD

Backend switching

CI/CD workflows are auto-generated from stack.toml by gen_config.py. Three backends are supported:

Backend	Workflows directory	When to use
gitea	.gitea/workflows/	Self-hosted Gitea instance (default)
github	.github/workflows/	GitHub repos / GitHub Actions
woodpecker	.woodpecker/	Woodpecker CI server

Switch backends by editing stack.toml:

[ci]
backend = "gitea"   # change to "github" or "woodpecker"

Then regenerate:

uv run python dev/scripts/gen_config.py          # generate new workflows
uv run python dev/scripts/gen_config.py --check   # verify (used in CI)
uv run python dev/scripts/gen_config.py --backend github  # one-off override

The generator:

1. Reads image definitions from stack.toml [images]
2. Dispatches to the active backend emitter (dev/scripts/backends/{backend}.py)
3. Writes workflow YAML files to the correct directory
4. Auto-removes stale workflows from inactive backend directories

Each backend has its own [ci.{backend}] config section in stack.toml with runner labels, registry endpoints, and authentication settings.

Pipelines

Six workflows generated per backend:

Pipeline	Trigger	What it does
ci.yml	Push/PR	Ruff lint + format, pytest (99% coverage), gen_config --check, skinny-install matrix
deploy.yml	Push to main	Build 5 images, Trivy scan, vuln reporting
harden.yml	Weekly cron / manual	Rebuild --no-cache, scan, auto-close/file vuln issues
rebuild-all.yml	Manual	Full rebuild + scan + deploy
infra-ci.yml	Path-filtered push	Hadolint Dockerfiles, dry-run builds
release.yml	Tag (v*)	uv build + release

Secret mapping

Secret	Gitea	GitHub	Woodpecker
Registry auth	REGISTRY_USER + REGISTRY_TOKEN	GITHUB_TOKEN (built-in)	registry_username + registry_password
Gitea API	GITEA_TOKEN	N/A	gitea_token
Vuln reporting	GITEA_TOKEN	GITHUB_TOKEN	gitea_token

Image tagging

Images are tagged with the short commit SHA (8 chars):

fhirworx/api:22ebc46e
fhirworx/notebooks:a4a2dcb0
fhirworx/docs:a4a2dcb0

Compose resolves via ${COMMIT_SHA:-latest} from .env. Custom images: api, notebooks, zotero, docs, mc.

Vulnerability management

The harden.yml pipeline rebuilds all images with --no-cache to pick up OS patches, runs Trivy scans, and auto-files or auto-closes Gitea issues via api.diag.vuln.

Credential management

All 19 service credentials derive from a single 256-bit root key via HKDF-SHA256. No passwords stored in .env — they regenerate deterministically from root key + commit SHA on each deploy.

ROOT_KEY + commit_sha → HKDF-SHA256 → all credentials → .env + backends

Tier	Salt	Rotates
Bootstrap	b"bootstrap"	Only when root key changes
Service	commit_sha	Every deploy

# Bootstrap (first time)
ROOT_KEY=$KEY uv run python -m api.auth bootstrap $(git rev-parse HEAD)

# Rotate (every deploy, automatic in CI)
ROOT_KEY=$KEY uv run python -m api.auth provision $COMMIT_SHA

# Dry-run
ROOT_KEY=$KEY uv run python -m api.auth derive $COMMIT_SHA --redact

Network segmentation

Network	Type	Services
gateway	external	Traefik, all public-facing
storage	internal	PostgreSQL, RustFS
data	internal	Nessie, Trino, Polaris
observability	internal	Grafana, Prometheus, Jaeger, Loki
ci	external	Woodpecker, Gitea, RustFS

Design system

Theme: loch (deep-navy). Traefik's inject-loch middleware rewrites HTML to inject loch.css and the favicon. Per-service CSS in assets/css/. Altair chart theme in assets/nature.py (Nature-conformant, accessible palette, IBM Plex Mono headings).

Data lakehouse

Component	Purpose
Nessie	Git-like branching and time travel for Iceberg tables
Polaris	Iceberg catalog with RBAC and multi-tenant governance
Trino	Distributed SQL (iceberg catalog)
RustFS	S3 storage backend (s3://lakehouse/)

Query via Trino CLI, JDBC, Web UI, PyIceberg, or DuckDB+PyArrow.

Project layout

stack/
├── compose.yml                  Docker Compose (22 services)
├── stack.toml                   Centralised configuration
├── pyproject.toml               Python project (uv, optional deps per module)
├── src/
│   ├── aco/                     ACO analytics (express, pipe, table, lake, load)
│   ├── api/                     FastAPI server + auth + diag
│   ├── bcda/                    BCDA FHIR R4 client
│   ├── bib/                     Zotero bibliography store, tags, metadata
│   ├── bls/                     BLS data
│   ├── ccw/                     CCW data dictionary
│   ├── cli/                     CLI entry point (typer)
│   ├── cms/                     CMS public data tables
│   ├── conf/                    Config loader, storage abstraction, table base
│   ├── pfs/                     Physician Fee Schedule
│   └── rex/                     REX fixed-width file processing
├── infra/                       Service configs and Dockerfiles
│   ├── images/                  All Dockerfiles (api, notebooks, zotero, docs, mc)
│   ├── traefik/                 Reverse proxy + loch CSS injection
│   ├── grafana/                 Dashboards and datasource provisioning
│   ├── prometheus/              Metrics collection
│   ├── loki/                    Log aggregation
│   └── ...                      coredns, nginx, trino, polaris, rustfs, act-runner, gitea
├── assets/                      Branding and styles
│   ├── css/                     Per-service CSS (dashboard, gitea, marimo, woodpecker)
│   ├── icons/                   Favicons, logos, SVGs
│   └── nature.py                Nature-conformant chart palette
├── cloud/                       Cloud provider deployment guides
│   ├── self-hosted/             Docker Compose (default)
│   ├── aws/                     S3, RDS, Glue, ECS
│   ├── gcp/                     GCS, Cloud SQL, BigQuery, Cloud Run
│   └── azure/                   ABFS, Azure SQL, Unity Catalog, Container Apps
├── dev/                         Dev tooling (scripts, hooks, seeds)
├── tests/                       11,955 tests at 99% coverage
├── notebooks/                   Marimo notebooks
├── docs/                        Docusaurus site
├── data/                        DuckDB, bib.sqlite, BCDA/CMS data (gitignored)
└── .env                         Derived credentials (not in git)

Prerequisites

• Docker (rootless mode)
• NVIDIA GPU with container toolkit
• Domain or /etc/hosts entries for *.homelab.fhirworx.io

# Fix for rootless NVIDIA containers
sudo sed -i 's/#no-cgroups = false/no-cgroups = true/' /etc/nvidia-container-runtime/config.toml

SSH access

# ~/.ssh/config
Host gitea
    HostName homelab.fhirworx.io
    Port 2222
    User git
    IdentityFile ~/.ssh/gitea_ed25519
    IdentitiesOnly yes