Some checks failed
CI / lint (push) Successful in 32s
CI / notebooks-smoke (push) Failing after 1m39s
Deploy / notebooks (push) Successful in 6m26s
Deploy / zotero (push) Has been skipped
Deploy / docs (push) Has been skipped
Deploy / api (push) Has been skipped
Deploy / mc (push) Has been skipped
Infra CI / notebooks (push) Successful in 48s
Infra CI / zotero (push) Successful in 27s
Infra CI / docs (push) Successful in 14s
Infra CI / api (push) Successful in 24s
Infra CI / mc (push) Successful in 13s
CI / test (push) Failing after 16m28s
Deploy / report (push) Successful in 16s
Three gates so notebook breakage can't ship or linger silently again (spec: docs/superpowers/specs/2026-07-09-notebook-quality-gates-design.md): 1. nb_integration.py: runs notebooks headless via 'marimo export session', parses the JSON snapshots for cell errors, emits a report, exits 1 on unexpected failures. New ci.yml notebooks-smoke job runs the data-independent [ci_smoke] set (infra/marimo/nb-tests.toml) on every push; new nightly notebooks-integration.yml runs the full set inside the prod container against real data, filing failures as issues. 2. nb_fe_smoke.py: headless-browser gate that loads the editor and fails on any console/page error — the test that would have blocked the 'd is not a constructor' bundle. Wired into infra-ci.yml after the notebooks image build (all traffic over the docker socket; -v bind mounts silently arrive empty in CI). Verified: exit 0 on the fixed image and live prod, exit 1 on a synthetic crashing page. 3. nb_issue_filer.py + nb-watcher compose sidecar: one issue per error signature (notebook + ename + normalized message), rate-limited recurrence comments, auto-close after 24h quiet — the watcher is the single closing authority. Tails container logs via the docker socket (stdlib unix-socket HTTP, stream demux) and parses live session snapshots. First production tick filed 9 real deduplicated issues (#546-#554: a real skin_subs_explorer bug, missing pyzotero/trino, nessie/api connectivity) under the new 'notebooks' label. Also: notebooks.Dockerfile now lets corepack honor marimo's pinned packageManager instead of 'pnpm@latest' — the last floating input to the frontend build after the lockfile fix. Tests: 6 new unit-test groups (snapshot parsing, signature stability, dedup decisions); full suite green including notebook-layout policy (config placed in infra/marimo/, not notebooks/).
124 lines
4.1 KiB
Python
124 lines
4.1 KiB
Python
"""Tests for dev/scripts/nb_issue_filer.py — signature stability and
|
|
dedup/auto-close decisions.
|
|
|
|
The filer exists because api.diag.ci files one issue per failure and never
|
|
closes them (23 duplicates accumulated for pkg-supply-chain alone). The
|
|
invariants under test: identical failures collapse to one signature across
|
|
runs, volatile details (paths, line numbers, addresses, counts) don't
|
|
change the signature, and the decide/close logic never files a duplicate
|
|
for an already-open signature.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import importlib.util
|
|
import sys
|
|
from pathlib import Path
|
|
|
|
_SCRIPT = Path(__file__).resolve().parents[2] / "dev" / "scripts" / "nb_issue_filer.py"
|
|
_spec = importlib.util.spec_from_file_location("_nb_issue_filer", _SCRIPT)
|
|
assert _spec and _spec.loader
|
|
filer = importlib.util.module_from_spec(_spec)
|
|
sys.modules["_nb_issue_filer"] = filer
|
|
_spec.loader.exec_module(filer)
|
|
|
|
|
|
# ── signature ────────────────────────────────────────────────────
|
|
|
|
|
|
def test_signature_stable_for_identical_error():
|
|
a = filer.signature("acodb_explorer.py", "exception", "division by zero")
|
|
b = filer.signature("acodb_explorer.py", "exception", "division by zero")
|
|
assert a == b
|
|
|
|
|
|
def test_signature_differs_across_notebooks_and_errors():
|
|
base = filer.signature("a.py", "exception", "boom")
|
|
assert filer.signature("b.py", "exception", "boom") != base
|
|
assert filer.signature("a.py", "interruption", "boom") != base
|
|
assert filer.signature("a.py", "exception", "other") != base
|
|
|
|
|
|
def test_signature_ignores_volatile_details():
|
|
a = filer.signature(
|
|
"nb.py",
|
|
"exception",
|
|
'IO Error: could not open "/tmp/marimo_2487488/cell_MJUe.py" at 0x7f3a2c1b'
|
|
" (attempt 3 of 5)",
|
|
)
|
|
b = filer.signature(
|
|
"nb.py",
|
|
"exception",
|
|
'IO Error: could not open "/tmp/marimo_9911223/cell_MJUe.py" at 0x559e00aa'
|
|
" (attempt 4 of 5)",
|
|
)
|
|
assert a == b
|
|
|
|
|
|
def test_signature_is_short_hex():
|
|
sig = filer.signature("nb.py", "exception", "x")
|
|
assert len(sig) == 12
|
|
int(sig, 16) # parses as hex
|
|
|
|
|
|
# ── normalize ────────────────────────────────────────────────────
|
|
|
|
|
|
def test_normalize_collapses_paths_hex_and_numbers():
|
|
s = filer.normalize_evalue(
|
|
"failed /home/kert/notebooks/x.py line 42 addr 0xDEADBEEF took 3.14s"
|
|
)
|
|
assert "0x" not in s.lower() or "0xN" in s
|
|
assert "/home/kert" not in s
|
|
assert "42" not in s
|
|
assert "3.14" not in s
|
|
|
|
|
|
def test_normalize_keeps_error_identity():
|
|
s = filer.normalize_evalue("Could not remove file: No such file or directory")
|
|
assert "no such file or directory" in s.lower()
|
|
|
|
|
|
# ── decide ───────────────────────────────────────────────────────
|
|
|
|
|
|
def test_decide_creates_when_no_open_issue():
|
|
action = filer.decide(existing=None, last_comment_age_s=None, cooldown_s=3600)
|
|
assert action == "create"
|
|
|
|
|
|
def test_decide_comments_after_cooldown():
|
|
action = filer.decide(
|
|
existing={"number": 7}, last_comment_age_s=7200, cooldown_s=3600
|
|
)
|
|
assert action == "comment"
|
|
|
|
|
|
def test_decide_skips_within_cooldown():
|
|
action = filer.decide(
|
|
existing={"number": 7}, last_comment_age_s=60, cooldown_s=3600
|
|
)
|
|
assert action == "skip"
|
|
|
|
|
|
# ── issue body marker round-trip ─────────────────────────────────
|
|
|
|
|
|
def test_marker_embeds_and_extracts_signature():
|
|
sig = filer.signature("nb.py", "exception", "boom")
|
|
body = filer.build_body(
|
|
notebook="nb.py",
|
|
ename="exception",
|
|
evalue="boom",
|
|
source="nightly-integration",
|
|
detail="Traceback ...",
|
|
sig=sig,
|
|
)
|
|
assert filer.extract_sig(body) == sig
|
|
assert "nb.py" in body
|
|
assert "nightly-integration" in body
|
|
|
|
|
|
def test_extract_sig_none_when_absent():
|
|
assert filer.extract_sig("no marker here") is None
|