Stack

Healthcare analytics platform on self-hosted infrastructure. Replaces dbt SQL models with narwhals DataFrame-agnostic expression functions, backed by DuckDB locally and Iceberg/Trino in the lakehouse. 22 services, 13 data pipelines, 11,931 tests at 99% coverage.

Quick start

# 1. Generate root key and seed .env
ROOT_KEY=$(openssl rand -hex 32)
cat > .env <<EOF
DOMAIN=homelab.fhirworx.io
HOST_IP=192.168.1.192
WOODPECKER_ADMIN=kert
EOF

# 2. Start core services and bootstrap credentials
docker compose up -d postgres rustfs traefik
ROOT_KEY=$ROOT_KEY uv run python -m api.auth bootstrap $(git rev-parse HEAD)

# 3. Start everything
docker compose up -d

# 4. Verify
curl -s http://localhost:8000/health | python3 -m json.tool

Services

All services route through Traefik at *.homelab.fhirworx.io.

Service	URL	Purpose
API	api (internal)	FastAPI — health, pipelines, bib endpoints
Notebooks	notebooks.homelab.fhirworx.io	Marimo notebooks (GPU-accelerated)
Docs	docs.homelab.fhirworx.io	Docusaurus — API reference + CMS bibliography
Gitea	gitea.homelab.fhirworx.io	Git server, container registry, LFS
Woodpecker CI	ci.homelab.fhirworx.io	CI/CD pipelines
Zotero	zotero.homelab.fhirworx.io	Reference manager (KasmVNC desktop)
Dashboard	homelab.fhirworx.io	Service launcher
RustFS	s3.homelab.fhirworx.io	S3-compatible object storage
Nessie	nessie.homelab.fhirworx.io	Git-like Iceberg catalog
Polaris	polaris.homelab.fhirworx.io	Iceberg catalog with governance
Trino	trino.homelab.fhirworx.io	Distributed SQL engine
Grafana	grafana.homelab.fhirworx.io	Dashboards
Prometheus	prometheus.homelab.fhirworx.io	Metrics
Jaeger	jaeger.homelab.fhirworx.io	Distributed tracing
Loki	loki.homelab.fhirworx.io	Log aggregation

Analytics platform

Architecture

  CCLF/BCDA/CMS data
        │
        ▼
  src/aco/express/     narwhals expressions (DataFrame-agnostic)
        │
        ▼
  src/aco/pipe/        pipeline runner with schema validation
        │
        ▼
  DuckDB (local)  ──or──  Iceberg + Trino (lakehouse)
        │
        ▼
  src/api/             FastAPI (health, runs, bib)
        │
        ▼
  notebooks/           Marimo interactive analysis

Pipelines

13 registered pipelines with 189 steps, validated by run_pipeline(steps, load) which enforces output schemas via pandera-style column checks.

Pipeline	Domain
input_layer	Raw CCLF/BCDA ingestion
core	Encounters, patients, providers
claims_preprocessing	Claim matching, encounter IDs
readmissions	30-day readmission rates
pharmacy	Part D drug utilization
ahrq_measures	AHRQ quality indicators
cms_quality_measures	CMS ACO quality metrics
hcc_suspecting	HCC risk adjustment
provider_attribution	Provider-patient assignment
quality_measures	Composite quality scores
data_quality	Validation and completeness
cclf	CCLF file parsing
main	Full pipeline orchestration

Expression layer

src/aco/express/ contains narwhals functions that work across polars, pandas, and cuDF. Each module mirrors a pipeline and exports pure transformation functions.

Configuration

stack.toml is the single config file. src/conf/ provides:

• cfg — attribute access (cfg.db.aco)
• path("db.aco") — absolute path resolution relative to repo root
• context() — environment switching (local/lake/databricks/trino)
• secret("api.secret", "STACK_API_SECRET") — env var with config fallback

Override context at runtime: STACK_CONTEXT=lake.

Bibliography

src/bib/ manages a Zotero-backed citation store in data/bib.sqlite. Column-level provenance via Tag.col(ref, desc). The docs site exports library.json for a searchable bibliography browser at /library.

CI/CD

Six Woodpecker pipelines in .woodpecker/:

Pipeline	Trigger	What it does
ci.yml	Push	Ruff lint, pytest, marimo check
deploy.yml	Push to main	Build images, Trivy scan, deploy, rotate credentials
harden.yml	Manual/cron	Rebuild all images --no-cache, scan, auto-close vuln issues
rebuild-all.yml	Manual	Full rebuild + scan + deploy
infra-ci.yml	Push	Hadolint Dockerfiles, build validation
release.yml	Tag	Release pipeline

Image tagging

Images are tagged with the short commit SHA (8 chars):

fhirworx/api:22ebc46e
fhirworx/notebooks:a4a2dcb0
fhirworx/docs:a4a2dcb0

Compose resolves via ${COMMIT_SHA:-latest} from .env. Custom images: api, notebooks, zotero, docs, mc.

Vulnerability management

The harden.yml pipeline rebuilds all images with --no-cache to pick up OS patches, runs Trivy scans, and auto-files or auto-closes Gitea issues via api.diag.vuln.

Credential management

All 19 service credentials derive from a single 256-bit root key via HKDF-SHA256. No passwords stored in .env — they regenerate deterministically from root key + commit SHA on each deploy.

ROOT_KEY + commit_sha → HKDF-SHA256 → all credentials → .env + backends

Tier	Salt	Rotates
Bootstrap	b"bootstrap"	Only when root key changes
Service	commit_sha	Every deploy

# Bootstrap (first time)
ROOT_KEY=$KEY uv run python -m api.auth bootstrap $(git rev-parse HEAD)

# Rotate (every deploy, automatic in CI)
ROOT_KEY=$KEY uv run python -m api.auth provision $COMMIT_SHA

# Dry-run
ROOT_KEY=$KEY uv run python -m api.auth derive $COMMIT_SHA --redact

Network segmentation

Network	Type	Services
gateway	external	Traefik, all public-facing
storage	internal	PostgreSQL, RustFS
data	internal	Nessie, Trino, Polaris
observability	internal	Grafana, Prometheus, Jaeger, Loki
ci	external	Woodpecker, Gitea, RustFS

Design system

Theme: loch (deep-navy). Traefik's inject-loch middleware rewrites HTML to inject loch.css and the favicon. Per-service CSS in styles/. Altair chart theme in styles/nature.py (Nature-conformant, accessible palette, IBM Plex Mono headings).

Data lakehouse

Component	Purpose
Nessie	Git-like branching and time travel for Iceberg tables
Polaris	Iceberg catalog with RBAC and multi-tenant governance
Trino	Distributed SQL (iceberg catalog)
RustFS	S3 storage backend (s3://lakehouse/)

Query via Trino CLI, JDBC, Web UI, PyIceberg, or DuckDB+PyArrow.

Project layout

stack/
├── compose.yml                  Docker Compose (22 services)
├── stack.toml                   Centralised configuration
├── pyproject.toml               Python project (uv)
├── src/
│   ├── aco/
│   │   ├── express/             narwhals transformation functions (15 modules)
│   │   ├── pipe/                pipeline runner + 13 pipeline modules
│   │   ├── table/               table schemas with column metadata
│   │   └── dag.py               pipeline DAG
│   ├── api/
│   │   ├── routes/              health, pipelines, bib endpoints
│   │   ├── auth/                HKDF credential derivation + provisioning
│   │   └── diag/                Trivy vuln reporter
│   ├── bib/                     Zotero store, tags, metadata
│   ├── conf/                    Config loader (cfg, path, context, secret)
│   ├── bcda/                    BCDA API client
│   ├── cms/                     CMS data utilities
│   ├── pfs/                     Physician Fee Schedule
│   └── rex/                     REX data processing
├── notebooks/                   Marimo notebooks (12 notebooks)
├── tests/                       11,931 tests at 99% coverage
├── styles/                      Loch design system (CSS, favicons, nature.py)
├── docs/                        Docusaurus site (griffe → Node → nginx)
├── .woodpecker/                 CI/CD pipelines (6 pipelines)
├── api/Dockerfile               FastAPI service
├── notebooks/Dockerfile         Marimo + CUDA 12.6 + cuDF
├── zotero/Dockerfile            Zotero desktop (KasmVNC + GPU)
├── docs/Dockerfile              3-stage: griffe → pnpm build → nginx:alpine
├── traefik/                     Reverse proxy config + loch injection
├── data/                        DuckDB, bib.sqlite, BCDA/CMS data
└── .env                         Derived credentials (not in git)

Prerequisites

• Docker (rootless mode)
• NVIDIA GPU with container toolkit
• Domain or /etc/hosts entries for *.homelab.fhirworx.io

# Fix for rootless NVIDIA containers
sudo sed -i 's/#no-cgroups = false/no-cgroups = true/' /etc/nvidia-container-runtime/config.toml

SSH access

# ~/.ssh/config
Host gitea
    HostName homelab.fhirworx.io
    Port 2222
    User git
    IdentityFile ~/.ssh/gitea_ed25519
    IdentitiesOnly yes