pfs: test_pipe_gaps fails under full-suite run — duckdb replacement-scan TypeError + multi-GB stdout spew #598

New Issue

kert · 2026-08-12T16:20:52Z

kert commented

2026-08-12 16:20:52 +00:00

6 tests in tests/pfs/test_pipe_gaps.py fail when the whole suite runs (uv run python -m pytest tests/ --no-cov, the pre-commit hook invocation) but pass in isolation. Root error: _duckdb.Error: TypeError: isinstance() arg 2 must be a type, a tuple of types, or a union at src/pfs/pipe.py:1764 (_insert_into, CREATE TABLE ... AS SELECT * FROM df WHERE 1=0). The same run wrote ~10 GB to pytest captured stdout before being killed (22 min wall, 2:38 CPU, state D). Suite was green at 3341d9e's hook run; duckdb 1.4.4 / polars 1.38.1 unchanged since 2026-07-17. Blocks every commit on the p36-cy2026-rule branch (P36).

6 tests in tests/pfs/test_pipe_gaps.py fail when the whole suite runs (`uv run python -m pytest tests/ --no-cov`, the pre-commit hook invocation) but pass in isolation. Root error: `_duckdb.Error: TypeError: isinstance() arg 2 must be a type, a tuple of types, or a union` at src/pfs/pipe.py:1764 (`_insert_into`, `CREATE TABLE ... AS SELECT * FROM df WHERE 1=0`). The same run wrote ~10 GB to pytest captured stdout before being killed (22 min wall, 2:38 CPU, state D). Suite was green at 3341d9e's hook run; duckdb 1.4.4 / polars 1.38.1 unchanged since 2026-07-17. Blocks every commit on the p36-cy2026-rule branch (P36).

kert added this to the P36: CY2026 PFS Proposed Rule — capture, ingest, APM analysis milestone 2026-08-12 16:20:52 +00:00

kert added the quality pfs labels 2026-08-12 16:20:52 +00:00

kert referenced this issue

2026-08-12 16:59:57 +00:00

ci: stale .git/hooks/pre-commit shadow copy misleads diagnosis — dev/hooks + sem.hooks are authoritative #599

kert commented

2026-08-12 17:13:39 +00:00

Fixed in 2c39737. Root cause: tests/pfs/test_pipe_gaps.py::TestLoadZipCarrierEdges::test_no_year_quarter_column patched pfs.pipe.pl.DataFrame — but pfs.pipe.pl IS the global polars module, so the test replaced polars.DataFrame process-wide with a MagicMock. When duckdb's Python replacement scan resolved the polars type for the first time inside that window, it cached the MagicMock permanently; every later SELECT * FROM df in the same process raised TypeError: isinstance() arg 2 must be a type... at pfs/pipe.py:1764. Order-dependent: single-file runs pass because an earlier test primes duckdb's cache with the real class; under the hook's scoped -n auto run the cold-cache worker hit the poison.

Fix: patch pfs.pipe.pl with a proxy types.ModuleType (real polars __dict__, only DataFrame overridden) so the global class is never touched.

Proof: (1) deterministic minimal repro both directions (cold process fails pre-fix, primed passes); (2) previously-failing 6 tests pass in the poisoned context post-fix; (3) full serial suite green: 14023 passed, 1 skipped, 26m45s. Also resolved en route: the multi-GB write volume attributed to this bug was actually distributed sqlite-journal churn from exercise-style tests (normal, if slow), and the stale .git/hooks/pre-commit copy that misdirected diagnosis was removed (#599).

Fixed in 2c39737. Root cause: `tests/pfs/test_pipe_gaps.py::TestLoadZipCarrierEdges::test_no_year_quarter_column` patched `pfs.pipe.pl.DataFrame` — but `pfs.pipe.pl` IS the global polars module, so the test replaced `polars.DataFrame` process-wide with a MagicMock. When duckdb's Python replacement scan resolved the polars type for the first time inside that window, it cached the MagicMock permanently; every later `SELECT * FROM df` in the same process raised `TypeError: isinstance() arg 2 must be a type...` at pfs/pipe.py:1764. Order-dependent: single-file runs pass because an earlier test primes duckdb's cache with the real class; under the hook's scoped `-n auto` run the cold-cache worker hit the poison. Fix: patch `pfs.pipe.pl` with a proxy `types.ModuleType` (real polars `__dict__`, only `DataFrame` overridden) so the global class is never touched. Proof: (1) deterministic minimal repro both directions (cold process fails pre-fix, primed passes); (2) previously-failing 6 tests pass in the poisoned context post-fix; (3) full serial suite green: 14023 passed, 1 skipped, 26m45s. Also resolved en route: the multi-GB write volume attributed to this bug was actually distributed sqlite-journal churn from exercise-style tests (normal, if slow), and the stale `.git/hooks/pre-commit` copy that misdirected diagnosis was removed (#599).

kert closed this issue

2026-08-12 17:13:39 +00:00

kert referenced this issue from a commit

2026-08-13 20:29:03 +00:00

fix(pfs): patch pfs.pipe.pl via proxy module — test poisoned global polars.DataFrame, corrupting duckdb's replacement-scan type cache (refs #598)

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: homelab/stack#598