feat(infra): containerise the BCDA manager — bcda image, compose service, Traefik route, dashboard tile
Some checks failed
CI / test (push) Failing after 2m35s
CI / notebooks-smoke (push) Successful in 1m34s
Infra CI / zotero (push) Successful in 16s
Infra CI / api (push) Successful in 17s
Infra CI / mc (push) Failing after 32s
Infra CI / docs (push) Successful in 18s
Infra CI / notebooks (push) Successful in 56s
Infra CI / llm (push) Successful in 19s
CI / lint (pull_request) Successful in 35s
Infra CI / bcda (push) Successful in 1m37s
CI / notebooks-smoke (pull_request) Successful in 3m6s
CI / test (pull_request) Failing after 4m52s
Infra CI / zotero (pull_request) Successful in 20s
Infra CI / docs (pull_request) Successful in 21s
Infra CI / api (pull_request) Successful in 19s
Infra CI / mc (pull_request) Failing after 39s
Infra CI / notebooks (pull_request) Successful in 1m18s
Infra CI / bcda (pull_request) Successful in 18s
Infra CI / llm (pull_request) Successful in 17s
CI / lint (push) Successful in 35s
Some checks failed
CI / test (push) Failing after 2m35s
CI / notebooks-smoke (push) Successful in 1m34s
Infra CI / zotero (push) Successful in 16s
Infra CI / api (push) Successful in 17s
Infra CI / mc (push) Failing after 32s
Infra CI / docs (push) Successful in 18s
Infra CI / notebooks (push) Successful in 56s
Infra CI / llm (push) Successful in 19s
CI / lint (pull_request) Successful in 35s
Infra CI / bcda (push) Successful in 1m37s
CI / notebooks-smoke (pull_request) Successful in 3m6s
CI / test (pull_request) Failing after 4m52s
Infra CI / zotero (pull_request) Successful in 20s
Infra CI / docs (pull_request) Successful in 21s
Infra CI / api (pull_request) Successful in 19s
Infra CI / mc (pull_request) Failing after 39s
Infra CI / notebooks (pull_request) Successful in 1m18s
Infra CI / bcda (pull_request) Successful in 18s
Infra CI / llm (pull_request) Successful in 17s
CI / lint (push) Successful in 35s
infra/images/bcda.Dockerfile follows api.Dockerfile (uv base, patched packages, curl for the healthcheck) with `uv sync --extra cli --extra bcda`; Streamlit's server settings live in infra/images/bcda/ streamlit.toml (headless, 0.0.0.0:8501, no usage stats, no file watcher); health is /_stcore/health. The compose service `bcda` sits on the gateway network, mounts ./data read-write (exports, aco.duckdb, replica publish) and ./dev read-only (vendored upstream, crosswalk seeds, vendor script), and takes BCDA_CLIENT_ID / BCDA_CLIENT_SECRET / BCDA_BASE_URL from .env with the public sandbox credentials as the default. Routed at bcda.<domain> behind git-sso with secure-headers, tiled on the dashboard, and added to [platform].subdomains (CoreDNS hosts and the SSO cert list follow) and [images] (deploy.yml builds, scans and pushes it on src/**, pyproject.toml or image-file changes; harden/rebuild-all/infra-ci regenerated accordingly). Built locally (2.15 GB), /_stcore/health -> ok, root -> 200; brought up on the host with compose. refs #1139
This commit is contained in:
@@ -32,3 +32,6 @@ assets/
|
||||
cloud/
|
||||
mirrors/
|
||||
.claude/
|
||||
# Streamlit server config COPYed by bcda.Dockerfile (repo-root context).
|
||||
!infra/images/bcda/
|
||||
!infra/images/bcda/**
|
||||
|
||||
@@ -15,6 +15,7 @@ jobs:
|
||||
zotero: ${{ steps.filter.outputs.zotero }}
|
||||
docs: ${{ steps.filter.outputs.docs }}
|
||||
api: ${{ steps.filter.outputs.api }}
|
||||
bcda: ${{ steps.filter.outputs.bcda }}
|
||||
llm: ${{ steps.filter.outputs.llm }}
|
||||
mc: ${{ steps.filter.outputs.mc }}
|
||||
steps:
|
||||
@@ -59,6 +60,7 @@ jobs:
|
||||
match zotero 'infra/images/zotero.Dockerfile' 'infra/images/zotero/*'
|
||||
match docs 'docs/*' 'infra/images/docs.Dockerfile'
|
||||
match api 'infra/images/api.Dockerfile' 'src/*' 'pyproject.toml'
|
||||
match bcda 'infra/images/bcda.Dockerfile' 'infra/images/bcda/*' 'src/*' 'pyproject.toml'
|
||||
match llm 'infra/images/llm.Dockerfile' 'src/*' 'pyproject.toml'
|
||||
match mc 'infra/rustfs/*' 'infra/images/mc.Dockerfile'
|
||||
|
||||
@@ -270,6 +272,58 @@ jobs:
|
||||
--sha "${{ github.sha }}" \
|
||||
--ref "${{ github.ref }}" || true
|
||||
|
||||
bcda:
|
||||
runs-on: ubuntu-latest
|
||||
needs: [changes]
|
||||
if: needs.changes.outputs.bcda == 'true'
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: https://github.com/actions/checkout@v4
|
||||
|
||||
- name: Install crane
|
||||
run: curl -sL https://github.com/google/go-containerregistry/releases/latest/download/go-containerregistry_Linux_x86_64.tar.gz | tar xz -C /usr/local/bin crane
|
||||
|
||||
- name: Log in to registry
|
||||
run: crane auth login git:3000 -u "${{ secrets.REGISTRY_USER }}" -p "${{ secrets.REGISTRY_TOKEN }}"
|
||||
env:
|
||||
CRANE_INSECURE: "true"
|
||||
|
||||
|
||||
- name: Install trivy
|
||||
run: curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sh -s -- -b /usr/local/bin
|
||||
|
||||
- name: Scan bcda
|
||||
run: trivy image --severity HIGH,CRITICAL --exit-code 0 --format json -o bcda-scan.json local/bcda:build
|
||||
|
||||
- name: Compute short SHA
|
||||
run: echo "SHORT_SHA=$(echo $GITHUB_SHA | head -c 8)" >> "$GITHUB_ENV"
|
||||
|
||||
- name: Build bcda
|
||||
run: docker build -f infra/images/bcda.Dockerfile -t local/bcda:build .
|
||||
|
||||
- name: Push bcda
|
||||
run: |
|
||||
docker save local/bcda:build -o /tmp/bcda.tar
|
||||
crane push /tmp/bcda.tar git:3000/homelab/stack/bcda:${{ env.SHORT_SHA }} --insecure
|
||||
crane push /tmp/bcda.tar git:3000/homelab/stack/bcda:latest --insecure
|
||||
|
||||
- name: File failure issue
|
||||
if: failure()
|
||||
env:
|
||||
GITEA_TOKEN: ${{ secrets.DEPLOY_TOKEN }}
|
||||
run: |
|
||||
# Jobs that never install uv (image builds, infra checks) still
|
||||
# need it here — otherwise the filer dies with "uv: command not
|
||||
# found" and the failure is never filed.
|
||||
command -v uv >/dev/null 2>&1 || curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
uv sync --no-dev --quiet 2>/dev/null || true
|
||||
uv run python -m api.diag.ci \
|
||||
--workflow "Deploy" --job "bcda" \
|
||||
--run "${{ github.run_number }}" \
|
||||
--sha "${{ github.sha }}" \
|
||||
--ref "${{ github.ref }}" || true
|
||||
|
||||
llm:
|
||||
runs-on: ubuntu-latest
|
||||
needs: [changes]
|
||||
@@ -370,7 +424,7 @@ jobs:
|
||||
|
||||
report:
|
||||
runs-on: ubuntu-latest
|
||||
needs: [changes, notebooks, zotero, docs, api, llm, mc]
|
||||
needs: [changes, notebooks, zotero, docs, api, bcda, llm, mc]
|
||||
if: always() && !cancelled()
|
||||
steps:
|
||||
- name: Checkout
|
||||
|
||||
@@ -67,6 +67,15 @@ jobs:
|
||||
crane push /tmp/api.tar git:3000/homelab/stack/api:hardened --insecure
|
||||
crane push /tmp/api.tar git:3000/homelab/stack/api:latest --insecure
|
||||
|
||||
- name: Build bcda
|
||||
run: docker build --no-cache -f infra/images/bcda.Dockerfile -t local/bcda:build .
|
||||
|
||||
- name: Push bcda
|
||||
run: |
|
||||
docker save local/bcda:build -o /tmp/bcda.tar
|
||||
crane push /tmp/bcda.tar git:3000/homelab/stack/bcda:hardened --insecure
|
||||
crane push /tmp/bcda.tar git:3000/homelab/stack/bcda:latest --insecure
|
||||
|
||||
- name: Build llm
|
||||
run: docker build --no-cache -f infra/images/llm.Dockerfile -t local/llm:build .
|
||||
|
||||
@@ -97,6 +106,9 @@ jobs:
|
||||
- name: Scan api
|
||||
run: trivy image --severity HIGH,CRITICAL --exit-code 0 --format json -o api-scan.json local/api:build
|
||||
|
||||
- name: Scan bcda
|
||||
run: trivy image --severity HIGH,CRITICAL --exit-code 0 --format json -o bcda-scan.json local/bcda:build
|
||||
|
||||
- name: Scan llm
|
||||
run: trivy image --severity HIGH,CRITICAL --exit-code 0 --format json -o llm-scan.json local/llm:build
|
||||
|
||||
@@ -105,7 +117,7 @@ jobs:
|
||||
GITEA_TOKEN: ${{ secrets.DEPLOY_TOKEN }}
|
||||
run: |
|
||||
uv sync --no-dev
|
||||
for f in notebooks-scan.json zotero-scan.json docs-scan.json api-scan.json llm-scan.json; do
|
||||
for f in notebooks-scan.json zotero-scan.json docs-scan.json api-scan.json bcda-scan.json llm-scan.json; do
|
||||
if [ -f "$f" ]; then
|
||||
uv run python -m api.diag.vuln --close "$f" || \
|
||||
uv run python -m api.diag.vuln "$f" || true
|
||||
|
||||
@@ -16,6 +16,10 @@ on:
|
||||
- 'infra/images/api.Dockerfile'
|
||||
- 'src/**'
|
||||
- 'pyproject.toml'
|
||||
- 'infra/images/bcda.Dockerfile'
|
||||
- 'infra/images/bcda/**'
|
||||
- 'src/**'
|
||||
- 'pyproject.toml'
|
||||
- 'infra/images/llm.Dockerfile'
|
||||
- 'src/**'
|
||||
- 'pyproject.toml'
|
||||
@@ -33,6 +37,10 @@ on:
|
||||
- 'infra/images/api.Dockerfile'
|
||||
- 'src/**'
|
||||
- 'pyproject.toml'
|
||||
- 'infra/images/bcda.Dockerfile'
|
||||
- 'infra/images/bcda/**'
|
||||
- 'src/**'
|
||||
- 'pyproject.toml'
|
||||
- 'infra/images/llm.Dockerfile'
|
||||
- 'src/**'
|
||||
- 'pyproject.toml'
|
||||
@@ -175,6 +183,39 @@ jobs:
|
||||
--sha "${{ github.sha }}" \
|
||||
--ref "${{ github.ref }}" || true
|
||||
|
||||
bcda:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: https://github.com/actions/checkout@v4
|
||||
|
||||
- name: Hadolint bcda
|
||||
uses: https://github.com/hadolint/hadolint-action@v3.1.0
|
||||
with:
|
||||
dockerfile: infra/images/bcda.Dockerfile
|
||||
|
||||
|
||||
|
||||
- name: Build bcda
|
||||
run: docker build -f infra/images/bcda.Dockerfile -t local/bcda:build .
|
||||
|
||||
- name: File failure issue
|
||||
if: failure()
|
||||
env:
|
||||
GITEA_TOKEN: ${{ secrets.DEPLOY_TOKEN }}
|
||||
run: |
|
||||
# Jobs that never install uv (image builds, infra checks) still
|
||||
# need it here — otherwise the filer dies with "uv: command not
|
||||
# found" and the failure is never filed.
|
||||
command -v uv >/dev/null 2>&1 || curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
uv sync --no-dev --quiet 2>/dev/null || true
|
||||
uv run python -m api.diag.ci \
|
||||
--workflow "Infra CI" --job "bcda" \
|
||||
--run "${{ github.run_number }}" \
|
||||
--sha "${{ github.sha }}" \
|
||||
--ref "${{ github.ref }}" || true
|
||||
|
||||
llm:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
|
||||
@@ -68,6 +68,15 @@ jobs:
|
||||
crane push /tmp/api.tar git:3000/homelab/stack/api:${{ env.SHORT_SHA }} --insecure
|
||||
crane push /tmp/api.tar git:3000/homelab/stack/api:latest --insecure
|
||||
|
||||
- name: Build bcda
|
||||
run: docker build -f infra/images/bcda.Dockerfile -t local/bcda:build .
|
||||
|
||||
- name: Push bcda
|
||||
run: |
|
||||
docker save local/bcda:build -o /tmp/bcda.tar
|
||||
crane push /tmp/bcda.tar git:3000/homelab/stack/bcda:${{ env.SHORT_SHA }} --insecure
|
||||
crane push /tmp/bcda.tar git:3000/homelab/stack/bcda:latest --insecure
|
||||
|
||||
- name: Build llm
|
||||
run: docker build -f infra/images/llm.Dockerfile -t local/llm:build .
|
||||
|
||||
@@ -98,6 +107,9 @@ jobs:
|
||||
- name: Scan api
|
||||
run: trivy image --severity HIGH,CRITICAL --exit-code 0 --format json -o api-scan.json local/api:build
|
||||
|
||||
- name: Scan bcda
|
||||
run: trivy image --severity HIGH,CRITICAL --exit-code 0 --format json -o bcda-scan.json local/bcda:build
|
||||
|
||||
- name: Scan llm
|
||||
run: trivy image --severity HIGH,CRITICAL --exit-code 0 --format json -o llm-scan.json local/llm:build
|
||||
|
||||
@@ -106,7 +118,7 @@ jobs:
|
||||
GITEA_TOKEN: ${{ secrets.DEPLOY_TOKEN }}
|
||||
run: |
|
||||
uv sync --no-dev
|
||||
for f in notebooks-scan.json zotero-scan.json docs-scan.json api-scan.json llm-scan.json; do
|
||||
for f in notebooks-scan.json zotero-scan.json docs-scan.json api-scan.json bcda-scan.json llm-scan.json; do
|
||||
[ -f "$f" ] && uv run python -m api.diag.vuln "$f" || true
|
||||
done
|
||||
|
||||
|
||||
29
compose.yml
29
compose.yml
@@ -669,6 +669,35 @@ services:
|
||||
- no-new-privileges:true
|
||||
restart: unless-stopped
|
||||
|
||||
# BCDA manager (P52): Streamlit over the bcda client, store, loader,
|
||||
# connector pipeline, crosswalk and vendored-upstream tooling. Routed at
|
||||
# bcda.fhirworx.io behind git-sso. Writes data/bcda (exports) and
|
||||
# aco.duckdb (flatten + load + input_layer) and republishes the replica,
|
||||
# so it mounts ./data read-write like the api; dev/ is mounted read-only
|
||||
# for the vendored upstream, the crosswalk seeds and the vendor script.
|
||||
# Public sandbox credentials are built in; BCDA_CLIENT_ID/SECRET override.
|
||||
bcda:
|
||||
image: ${IMAGE_PREFIX:-fhirworx}/bcda:${COMMIT_SHA:-latest}
|
||||
pull_policy: if_not_present
|
||||
build:
|
||||
context: .
|
||||
dockerfile: infra/images/bcda.Dockerfile
|
||||
container_name: bcda
|
||||
networks:
|
||||
- gateway
|
||||
environment:
|
||||
- BCDA_CLIENT_ID=${BCDA_CLIENT_ID:-}
|
||||
- BCDA_CLIENT_SECRET=${BCDA_CLIENT_SECRET:-}
|
||||
- BCDA_BASE_URL=${BCDA_BASE_URL:-https://sandbox.bcda.cms.gov}
|
||||
volumes:
|
||||
- ./data:/app/data
|
||||
- ./dev:/app/dev:ro
|
||||
labels:
|
||||
- "promtail=true"
|
||||
security_opt:
|
||||
- no-new-privileges:true
|
||||
restart: unless-stopped
|
||||
|
||||
# SSO-guarded RAG chat UI (P34). Routed at llm.fhirworx.io via Traefik's
|
||||
# git-sso middleware ($reef in infra/traefik/dynamic/services.yml). Joins
|
||||
# gateway (Traefik), storage (postgres:5432 / pgvector), and data
|
||||
|
||||
@@ -78,3 +78,7 @@ Decisions:
|
||||
6. **Launch:** `stack bcda ui` (port, headless flags) runs `streamlit run src/bcda/ui/app.py`; Streamlit lives in the `bcda` extra. Containerising it (image + traefik route + dashboard tile) is deliberately left out of this slice.
|
||||
|
||||
Tests: service with `httpx.MockTransport`, runner for thread-routed logs and state ordering, every page through `AppTest.from_string` with the service stubbed (buttons, forms, error paths), the app shell through `AppTest.from_file`, and the CLI through `CliRunner` — all at 100 % line coverage.
|
||||
|
||||
### Containerised (2026-10-25)
|
||||
|
||||
`infra/images/bcda.Dockerfile` (same base and install pattern as the api image; `uv sync --extra cli --extra bcda`, Streamlit settings in `infra/images/bcda/streamlit.toml`, health at `/_stcore/health`), compose service `bcda` on the gateway network with `./data` read-write (exports, aco.duckdb, replica publish) and `./dev` read-only (vendored upstream, crosswalk seeds, vendor script), Traefik route `bcda.<domain>` behind git-sso, a dashboard tile, the `bcda` subdomain in stack.toml (CoreDNS hosts and the SSO cert list follow) and an `[images.bcda]` entry so the deploy workflow builds, scans and pushes it on changes to `src/**`, `pyproject.toml` or the image files. Public sandbox credentials are built in; `BCDA_CLIENT_ID` / `BCDA_CLIENT_SECRET` / `BCDA_BASE_URL` in `.env` override.
|
||||
|
||||
@@ -17,3 +17,4 @@
|
||||
192.168.1.192 s3.fhirworx.io
|
||||
192.168.1.192 s3console.fhirworx.io
|
||||
192.168.1.192 llm.fhirworx.io
|
||||
192.168.1.192 bcda.fhirworx.io
|
||||
|
||||
36
infra/images/bcda.Dockerfile
Normal file
36
infra/images/bcda.Dockerfile
Normal file
@@ -0,0 +1,36 @@
|
||||
# syntax=docker/dockerfile:1
|
||||
# BCDA manager — Streamlit app over the bcda client, store, loader,
|
||||
# connector pipeline, crosswalk and vendored-upstream tooling
|
||||
# (src/bcda/ui, `stack bcda ui`). Same base and install pattern as
|
||||
# api.Dockerfile; the bcda extra brings streamlit, the cli bundle brings
|
||||
# aco (flatten + load + pipeline) and conf.
|
||||
FROM ghcr.io/astral-sh/uv:python3.13-bookworm-slim
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
ARG PYPI_INDEX_URL=""
|
||||
|
||||
RUN apt-get update && apt-get upgrade -y && apt-get install -y --no-install-recommends curl && rm -rf /var/lib/apt/lists/*
|
||||
|
||||
COPY pyproject.toml uv.lock README.md ./
|
||||
COPY src/ src/
|
||||
|
||||
ENV UV_PYTHON_PREFERENCE=only-system \
|
||||
UV_LINK_MODE=copy \
|
||||
UV_PROJECT_ENVIRONMENT=.venv \
|
||||
UV_INDEX_URL=${PYPI_INDEX_URL}
|
||||
RUN uv sync --no-dev --extra cli --extra bcda && uv pip install -e .
|
||||
|
||||
COPY stack.toml ./
|
||||
|
||||
# Streamlit reads its server settings from here (headless, no usage
|
||||
# stats, bound to all interfaces, websocket-friendly behind Traefik).
|
||||
COPY infra/images/bcda/streamlit.toml /app/.streamlit/config.toml
|
||||
|
||||
EXPOSE 8501
|
||||
|
||||
# /_stcore/health answers "ok" once the Streamlit server is up.
|
||||
HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
|
||||
CMD curl -sf --max-time 4 http://localhost:8501/_stcore/health || exit 1
|
||||
|
||||
CMD ["uv", "run", "--no-sync", "streamlit", "run", "src/bcda/ui/app.py"]
|
||||
15
infra/images/bcda/streamlit.toml
Normal file
15
infra/images/bcda/streamlit.toml
Normal file
@@ -0,0 +1,15 @@
|
||||
# Streamlit server config for the containerised BCDA manager.
|
||||
[server]
|
||||
headless = true
|
||||
address = "0.0.0.0"
|
||||
port = 8501
|
||||
enableXsrfProtection = true
|
||||
enableCORS = false
|
||||
fileWatcherType = "none"
|
||||
runOnSave = false
|
||||
|
||||
[browser]
|
||||
gatherUsageStats = false
|
||||
|
||||
[theme]
|
||||
base = "light"
|
||||
@@ -42,6 +42,16 @@
|
||||
<div class="stripe"></div>
|
||||
</a>
|
||||
|
||||
<a data-subdomain="bcda" data-path="/_stcore/health" target="_blank" class="tile notebook">
|
||||
<span class="status"></span>
|
||||
<span class="icon">🏥</span>
|
||||
<div class="tile-eyebrow">Claims</div>
|
||||
<div class="tile-title">BCDA Manager</div>
|
||||
<p class="tile-desc">Beneficiary Claims Data API v3 — exports, store, load, crosswalk</p>
|
||||
<span class="tile-port"></span>
|
||||
<div class="stripe"></div>
|
||||
</a>
|
||||
|
||||
<a data-subdomain="traefik" target="_blank" class="tile proxy">
|
||||
<span class="status"></span>
|
||||
<span class="icon">🔗</span>
|
||||
|
||||
@@ -8,6 +8,7 @@
|
||||
"webdav" (dict "port" "8080" "theme" false "mw" "secure-headers")
|
||||
"api" (dict "port" "8000" "theme" false "mw" "secure-headers")
|
||||
"llm" (dict "port" "8000" "theme" false "mw" "git-sso,secure-headers")
|
||||
"bcda" (dict "port" "8501" "theme" false "mw" "git-sso,secure-headers")
|
||||
"nessie" (dict "port" "19120" "theme" false "mw" "git-sso,infra-headers")
|
||||
"trino" (dict "port" "8080" "theme" true "mw" "git-sso,infra-headers")
|
||||
"polaris" (dict "port" "8181" "theme" false "mw" "git-sso,infra-headers")
|
||||
|
||||
@@ -24,7 +24,7 @@ subdomains = [
|
||||
"dashboard", "docs", "git", "ci", "notebooks", "zotero",
|
||||
"webdav", "api", "nessie", "trino", "polaris",
|
||||
"grafana", "prometheus", "tempo", "loki",
|
||||
"s3", "s3console", "llm",
|
||||
"s3", "s3console", "llm", "bcda",
|
||||
]
|
||||
|
||||
[services]
|
||||
@@ -311,6 +311,12 @@ dockerfile = "infra/images/api.Dockerfile"
|
||||
context = "."
|
||||
path_filter = ["infra/images/api.Dockerfile", "src/**", "pyproject.toml"]
|
||||
|
||||
# BCDA manager (Streamlit, src/bcda/ui) — P52.
|
||||
[images.bcda]
|
||||
dockerfile = "infra/images/bcda.Dockerfile"
|
||||
context = "."
|
||||
path_filter = ["infra/images/bcda.Dockerfile", "infra/images/bcda/**", "src/**", "pyproject.toml"]
|
||||
|
||||
[images.llm]
|
||||
dockerfile = "infra/images/llm.Dockerfile"
|
||||
context = "."
|
||||
|
||||
Reference in New Issue
Block a user