stack/.gitea/workflows/deploy.yml at 3b6144609e5c07de0bf619616414db0ae0012f3b

Files

kert 41c7d318e9 fix(ci): make image and package scanning see again

Harden has failed every Sunday since 2026-09-27: the mc build (dead
quay.io/minio/mc) aborted the job before a single trivy step ran, so
for a month no image was scanned and nothing said so beyond an
auto-filed issue per week.

Harden (gen_config -> .gitea/workflows/harden.yml)
- Each image is built -> scanned -> pushed on its own. A failed build
  is continue-on-error and skips only that image's scan and push; a
  gate step after the vuln report fails the job, so the breakage is
  still filed, never swallowed.
- Builds use --pull --no-cache: --no-cache alone reused whatever stale
  base the runner had cached.
- mc is scanned now too (it was built but excluded).

Tooling pins (harden, deploy, rebuild-all, pkg-supply-chain)
- trivy 0.76.0 via the tagged installer (it checks the release sha256)
  instead of install.sh from `main`.
- crane 0.22.1 with its sha256 checked against the release
  checksums.txt, instead of an unverified `releases/latest` tarball.
- Both are Renovate-tracked constants in backends/gitea.py.

Package supply chain
- docs/pnpm-lock.yaml is scanned nightly into data/pkg-vulns-docs.json
  and filed like uv.lock findings.
- pkg_issues.py read only the first 50 open `ci` issues, so once CI
  failures piled up every known CVE past page one was invisible: re-filed
  nightly (174 open issues for 91 titles) and never closed. It now walks
  every page, closes all duplicates of a title, and runs the close pass
  even when a scan comes back clean — it used to return early there,
  i.e. exactly when the most issues resolve. Dry run against the live
  tracker with the new lock: closes all 174, files none.

Refs #1365

2026-10-27 11:04:54 -04:00

440 lines

17 KiB

YAML

Raw Blame History

 # DO NOT EDIT — generated by gen_config.py from stack.toml
 # Re-generate: uv run python dev/scripts/gen_config.py
 name: Deploy
 on:
   push:
     branches: [main]
 jobs:
   changes:
     runs-on: ubuntu-latest
     outputs:
       notebooks: ${{ steps.filter.outputs.notebooks }}
       zotero: ${{ steps.filter.outputs.zotero }}
       docs: ${{ steps.filter.outputs.docs }}
       api: ${{ steps.filter.outputs.api }}
       llm: ${{ steps.filter.outputs.llm }}
       mc: ${{ steps.filter.outputs.mc }}
     steps:
       - name: Checkout
         uses: https://github.com/actions/checkout@v4
       - name: Detect changed paths
         id: filter
         env:
           BEFORE: ${{ github.event.before }}
         run: |
           # `github.event.head_commit.modified` only describes the tip commit
           # and is empty for merge commits, so a PR merged with a merge commit
           # never matched any path filter. Diff the pushed range instead:
           # the branch tip before the push against the new one. When that
           # previous tip is unknown or unfetchable (first push, history
           # rewrite) build everything rather than nothing.
           changed=__all__
           if [ -n "$BEFORE" ] && [ "$BEFORE" != "0000000000000000000000000000000000000000" ] \
              && git fetch --quiet --depth=1 origin "$BEFORE"; then
             changed="$(git diff --name-only "$BEFORE" "$GITHUB_SHA")" || changed=__all__
           fi
           echo "changed since $BEFORE:"
           printf '%s\n' "$changed"
           match() {
             name="$1"; shift
             hit=false
             if [ "$changed" = __all__ ]; then
               hit=true
             else
               while IFS= read -r f; do
                 [ -n "$f" ] || continue
                 for g in "$@"; do
                   case "$f" in $g) hit=true ;; esac
                 done
               done <<< "$changed"
             fi
             echo "$name=$hit" >> "$GITHUB_OUTPUT"
             echo "$name: $hit"
           }
           match notebooks 'notebooks/*' 'infra/marimo/*' 'infra/images/notebooks.Dockerfile'
           match zotero 'infra/images/zotero.Dockerfile' 'infra/images/zotero/*'
           match docs 'docs/*' 'infra/images/docs.Dockerfile'
           match api 'infra/images/api.Dockerfile' 'src/*' 'pyproject.toml'
           match llm 'infra/images/llm.Dockerfile' 'src/*' 'pyproject.toml'
           match mc 'infra/rustfs/*' 'infra/images/mc.Dockerfile'
   notebooks:
     runs-on: ubuntu-latest
     needs: [changes]
     if: needs.changes.outputs.notebooks == 'true'
     steps:
       - name: Checkout
         uses: https://github.com/actions/checkout@v4
       - name: Install crane
         run: |
           cd /tmp
           base=https://github.com/google/go-containerregistry/releases/download/v0.22.1
           tgz=go-containerregistry_Linux_x86_64.tar.gz
           curl -fsSLO "$base/$tgz"
           curl -fsSL "$base/checksums.txt" | grep -F "  $tgz" | sha256sum -c -
           tar xzf "$tgz" -C /usr/local/bin crane
       - name: Log in to registry
         run: crane auth login git:3000 -u "${{ secrets.REGISTRY_USER }}" -p "${{ secrets.REGISTRY_TOKEN }}"
         env:
           CRANE_INSECURE: "true"
       - name: Install trivy
         run: curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/v0.76.0/contrib/install.sh | sh -s -- -b /usr/local/bin v0.76.0
       - name: Scan notebooks
         run: trivy image --severity HIGH,CRITICAL --exit-code 0 --format json -o notebooks-scan.json local/notebooks:build
       - name: Compute short SHA
         run: echo "SHORT_SHA=$(echo $GITHUB_SHA | head -c 8)" >> "$GITHUB_ENV"
       - name: Build notebooks
         run: docker build -f infra/images/notebooks.Dockerfile -t local/notebooks:build .
       - name: Push notebooks
         run: |
           docker save local/notebooks:build -o /tmp/notebooks.tar
           crane push /tmp/notebooks.tar git:3000/homelab/stack/notebooks:${{ env.SHORT_SHA }} --insecure
           crane push /tmp/notebooks.tar git:3000/homelab/stack/notebooks:latest --insecure
       - name: File failure issue
         if: failure()
         env:
           GITEA_TOKEN: ${{ secrets.DEPLOY_TOKEN }}
         run: |
           # Jobs that never install uv (image builds, infra checks) still
           # need it here — otherwise the filer dies with "uv: command not
           # found" and the failure is never filed.
           command -v uv >/dev/null 2>&1 || curl -LsSf https://astral.sh/uv/install.sh | sh
           export PATH="$HOME/.local/bin:$PATH"
           uv sync --no-dev --quiet 2>/dev/null || true
           uv run python -m api.diag.ci \
             --workflow "Deploy" --job "notebooks" \
             --run "${{ github.run_number }}" \
             --sha "${{ github.sha }}" \
             --ref "${{ github.ref }}" || true
   zotero:
     runs-on: ubuntu-latest
     needs: [changes]
     if: needs.changes.outputs.zotero == 'true'
     steps:
       - name: Checkout
         uses: https://github.com/actions/checkout@v4
       - name: Install crane
         run: |
           cd /tmp
           base=https://github.com/google/go-containerregistry/releases/download/v0.22.1
           tgz=go-containerregistry_Linux_x86_64.tar.gz
           curl -fsSLO "$base/$tgz"
           curl -fsSL "$base/checksums.txt" | grep -F "  $tgz" | sha256sum -c -
           tar xzf "$tgz" -C /usr/local/bin crane
       - name: Log in to registry
         run: crane auth login git:3000 -u "${{ secrets.REGISTRY_USER }}" -p "${{ secrets.REGISTRY_TOKEN }}"
         env:
           CRANE_INSECURE: "true"
       - name: Install trivy
         run: curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/v0.76.0/contrib/install.sh | sh -s -- -b /usr/local/bin v0.76.0
       - name: Scan zotero
         run: trivy image --severity HIGH,CRITICAL --exit-code 0 --format json -o zotero-scan.json local/zotero:build
       - name: Compute short SHA
         run: echo "SHORT_SHA=$(echo $GITHUB_SHA | head -c 8)" >> "$GITHUB_ENV"
       - name: Build zotero
         run: docker build -f infra/images/zotero.Dockerfile -t local/zotero:build .
       - name: Push zotero
         run: |
           docker save local/zotero:build -o /tmp/zotero.tar
           crane push /tmp/zotero.tar git:3000/homelab/stack/zotero:${{ env.SHORT_SHA }} --insecure
           crane push /tmp/zotero.tar git:3000/homelab/stack/zotero:latest --insecure
       - name: File failure issue
         if: failure()
         env:
           GITEA_TOKEN: ${{ secrets.DEPLOY_TOKEN }}
         run: |
           # Jobs that never install uv (image builds, infra checks) still
           # need it here — otherwise the filer dies with "uv: command not
           # found" and the failure is never filed.
           command -v uv >/dev/null 2>&1 || curl -LsSf https://astral.sh/uv/install.sh | sh
           export PATH="$HOME/.local/bin:$PATH"
           uv sync --no-dev --quiet 2>/dev/null || true
           uv run python -m api.diag.ci \
             --workflow "Deploy" --job "zotero" \
             --run "${{ github.run_number }}" \
             --sha "${{ github.sha }}" \
             --ref "${{ github.ref }}" || true
   docs:
     runs-on: ubuntu-latest
     needs: [changes]
     if: needs.changes.outputs.docs == 'true'
     steps:
       - name: Checkout
         uses: https://github.com/actions/checkout@v4
       - name: Install crane
         run: |
           cd /tmp
           base=https://github.com/google/go-containerregistry/releases/download/v0.22.1
           tgz=go-containerregistry_Linux_x86_64.tar.gz
           curl -fsSLO "$base/$tgz"
           curl -fsSL "$base/checksums.txt" | grep -F "  $tgz" | sha256sum -c -
           tar xzf "$tgz" -C /usr/local/bin crane
       - name: Log in to registry
         run: crane auth login git:3000 -u "${{ secrets.REGISTRY_USER }}" -p "${{ secrets.REGISTRY_TOKEN }}"
         env:
           CRANE_INSECURE: "true"
       - name: Install trivy
         run: curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/v0.76.0/contrib/install.sh | sh -s -- -b /usr/local/bin v0.76.0
       - name: Scan docs
         run: trivy image --severity HIGH,CRITICAL --exit-code 0 --format json -o docs-scan.json local/docs:build
       - name: Compute short SHA
         run: echo "SHORT_SHA=$(echo $GITHUB_SHA | head -c 8)" >> "$GITHUB_ENV"
       - name: Build docs
         run: docker build -f infra/images/docs.Dockerfile -t local/docs:build .
       - name: Push docs
         run: |
           docker save local/docs:build -o /tmp/docs.tar
           crane push /tmp/docs.tar git:3000/homelab/stack/docs:${{ env.SHORT_SHA }} --insecure
           crane push /tmp/docs.tar git:3000/homelab/stack/docs:latest --insecure
       - name: File failure issue
         if: failure()
         env:
           GITEA_TOKEN: ${{ secrets.DEPLOY_TOKEN }}
         run: |
           # Jobs that never install uv (image builds, infra checks) still
           # need it here — otherwise the filer dies with "uv: command not
           # found" and the failure is never filed.
           command -v uv >/dev/null 2>&1 || curl -LsSf https://astral.sh/uv/install.sh | sh
           export PATH="$HOME/.local/bin:$PATH"
           uv sync --no-dev --quiet 2>/dev/null || true
           uv run python -m api.diag.ci \
             --workflow "Deploy" --job "docs" \
             --run "${{ github.run_number }}" \
             --sha "${{ github.sha }}" \
             --ref "${{ github.ref }}" || true
   api:
     runs-on: ubuntu-latest
     needs: [changes]
     if: needs.changes.outputs.api == 'true'
     steps:
       - name: Checkout
         uses: https://github.com/actions/checkout@v4
       - name: Install crane
         run: |
           cd /tmp
           base=https://github.com/google/go-containerregistry/releases/download/v0.22.1
           tgz=go-containerregistry_Linux_x86_64.tar.gz
           curl -fsSLO "$base/$tgz"
           curl -fsSL "$base/checksums.txt" | grep -F "  $tgz" | sha256sum -c -
           tar xzf "$tgz" -C /usr/local/bin crane
       - name: Log in to registry
         run: crane auth login git:3000 -u "${{ secrets.REGISTRY_USER }}" -p "${{ secrets.REGISTRY_TOKEN }}"
         env:
           CRANE_INSECURE: "true"
       - name: Install trivy
         run: curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/v0.76.0/contrib/install.sh | sh -s -- -b /usr/local/bin v0.76.0
       - name: Scan api
         run: trivy image --severity HIGH,CRITICAL --exit-code 0 --format json -o api-scan.json local/api:build
       - name: Compute short SHA
         run: echo "SHORT_SHA=$(echo $GITHUB_SHA | head -c 8)" >> "$GITHUB_ENV"
       - name: Build api
         run: docker build -f infra/images/api.Dockerfile -t local/api:build .
       - name: Push api
         run: |
           docker save local/api:build -o /tmp/api.tar
           crane push /tmp/api.tar git:3000/homelab/stack/api:${{ env.SHORT_SHA }} --insecure
           crane push /tmp/api.tar git:3000/homelab/stack/api:latest --insecure
       - name: File failure issue
         if: failure()
         env:
           GITEA_TOKEN: ${{ secrets.DEPLOY_TOKEN }}
         run: |
           # Jobs that never install uv (image builds, infra checks) still
           # need it here — otherwise the filer dies with "uv: command not
           # found" and the failure is never filed.
           command -v uv >/dev/null 2>&1 || curl -LsSf https://astral.sh/uv/install.sh | sh
           export PATH="$HOME/.local/bin:$PATH"
           uv sync --no-dev --quiet 2>/dev/null || true
           uv run python -m api.diag.ci \
             --workflow "Deploy" --job "api" \
             --run "${{ github.run_number }}" \
             --sha "${{ github.sha }}" \
             --ref "${{ github.ref }}" || true
   llm:
     runs-on: ubuntu-latest
     needs: [changes]
     if: needs.changes.outputs.llm == 'true'
     steps:
       - name: Checkout
         uses: https://github.com/actions/checkout@v4
       - name: Install crane
         run: |
           cd /tmp
           base=https://github.com/google/go-containerregistry/releases/download/v0.22.1
           tgz=go-containerregistry_Linux_x86_64.tar.gz
           curl -fsSLO "$base/$tgz"
           curl -fsSL "$base/checksums.txt" | grep -F "  $tgz" | sha256sum -c -
           tar xzf "$tgz" -C /usr/local/bin crane
       - name: Log in to registry
         run: crane auth login git:3000 -u "${{ secrets.REGISTRY_USER }}" -p "${{ secrets.REGISTRY_TOKEN }}"
         env:
           CRANE_INSECURE: "true"
       - name: Install trivy
         run: curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/v0.76.0/contrib/install.sh | sh -s -- -b /usr/local/bin v0.76.0
       - name: Scan llm
         run: trivy image --severity HIGH,CRITICAL --exit-code 0 --format json -o llm-scan.json local/llm:build
       - name: Compute short SHA
         run: echo "SHORT_SHA=$(echo $GITHUB_SHA | head -c 8)" >> "$GITHUB_ENV"
       - name: Build llm
         run: docker build -f infra/images/llm.Dockerfile -t local/llm:build .
       - name: Push llm
         run: |
           docker save local/llm:build -o /tmp/llm.tar
           crane push /tmp/llm.tar git:3000/homelab/stack/llm:${{ env.SHORT_SHA }} --insecure
           crane push /tmp/llm.tar git:3000/homelab/stack/llm:latest --insecure
       - name: File failure issue
         if: failure()
         env:
           GITEA_TOKEN: ${{ secrets.DEPLOY_TOKEN }}
         run: |
           # Jobs that never install uv (image builds, infra checks) still
           # need it here — otherwise the filer dies with "uv: command not
           # found" and the failure is never filed.
           command -v uv >/dev/null 2>&1 || curl -LsSf https://astral.sh/uv/install.sh | sh
           export PATH="$HOME/.local/bin:$PATH"
           uv sync --no-dev --quiet 2>/dev/null || true
           uv run python -m api.diag.ci \
             --workflow "Deploy" --job "llm" \
             --run "${{ github.run_number }}" \
             --sha "${{ github.sha }}" \
             --ref "${{ github.ref }}" || true
   mc:
     runs-on: ubuntu-latest
     needs: [changes]
     if: needs.changes.outputs.mc == 'true'
     steps:
       - name: Checkout
         uses: https://github.com/actions/checkout@v4
       - name: Install crane
         run: |
           cd /tmp
           base=https://github.com/google/go-containerregistry/releases/download/v0.22.1
           tgz=go-containerregistry_Linux_x86_64.tar.gz
           curl -fsSLO "$base/$tgz"
           curl -fsSL "$base/checksums.txt" | grep -F "  $tgz" | sha256sum -c -
           tar xzf "$tgz" -C /usr/local/bin crane
       - name: Log in to registry
         run: crane auth login git:3000 -u "${{ secrets.REGISTRY_USER }}" -p "${{ secrets.REGISTRY_TOKEN }}"
         env:
           CRANE_INSECURE: "true"
       - name: Install trivy
         run: curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/v0.76.0/contrib/install.sh | sh -s -- -b /usr/local/bin v0.76.0
       - name: Scan mc
         run: trivy image --severity HIGH,CRITICAL --exit-code 0 --format json -o mc-scan.json local/mc:build
       - name: Compute short SHA
         run: echo "SHORT_SHA=$(echo $GITHUB_SHA | head -c 8)" >> "$GITHUB_ENV"
       - name: Build mc
         run: docker build -f infra/images/mc.Dockerfile -t local/mc:build infra/rustfs/
       - name: Push mc
         run: |
           docker save local/mc:build -o /tmp/mc.tar
           crane push /tmp/mc.tar git:3000/homelab/stack/mc:${{ env.SHORT_SHA }} --insecure
           crane push /tmp/mc.tar git:3000/homelab/stack/mc:latest --insecure
       - name: File failure issue
         if: failure()
         env:
           GITEA_TOKEN: ${{ secrets.DEPLOY_TOKEN }}
         run: |
           # Jobs that never install uv (image builds, infra checks) still
           # need it here — otherwise the filer dies with "uv: command not
           # found" and the failure is never filed.
           command -v uv >/dev/null 2>&1 || curl -LsSf https://astral.sh/uv/install.sh | sh
           export PATH="$HOME/.local/bin:$PATH"
           uv sync --no-dev --quiet 2>/dev/null || true
           uv run python -m api.diag.ci \
             --workflow "Deploy" --job "mc" \
             --run "${{ github.run_number }}" \
             --sha "${{ github.sha }}" \
             --ref "${{ github.ref }}" || true
   report:
     runs-on: ubuntu-latest
     needs: [changes, notebooks, zotero, docs, api, llm, mc]
     if: always() && !cancelled()
     steps:
       - name: Checkout
         uses: https://github.com/actions/checkout@v4
       - name: Set up uv
         run: curl -LsSf https://astral.sh/uv/install.sh | sh
         env:
           UV_INSTALL_DIR: /usr/local/bin
       - name: Report
         # Only file an issue if at least one upstream build job failed.
         # Without this gate api.diag.ci runs unconditionally and files
         # a fake "failed" issue on every successful Deploy run.
         if: contains(needs.*.result, 'failure')
         env:
           GITEA_TOKEN: ${{ secrets.DEPLOY_TOKEN }}
         run: |
           uv sync --no-dev --quiet 2>/dev/null || true
           uv run python -m api.diag.ci \
             --workflow "Deploy" --job "report" \
             --run "${{ github.run_number }}" \
             --sha "${{ github.sha }}" \
             --ref "${{ github.ref }}" || true

440 lines 17 KiB YAML Raw Blame History

440 lines

17 KiB

YAML

Raw Blame History