notebooks: code_families.py sections 5 and 7b crash in the notebooks container (llm package needs langchain; LLM_DB_PASSWORD not provided) #720

New Issue

kert · 2026-09-11T14:27:26Z

kert commented

2026-09-11 14:27:26 +00:00

Symptom (notebooks container, marimo://code_families.py cell-14 and the 7b cell):

from llm.config import load as _load_llm_cfg
File /home/kert/src/llm/__init__.py, line 21: from llm.pool import HostPool
File /home/kert/src/llm/pool.py, line 22: from langchain_core.embeddings import Embeddings
ModuleNotFoundError: No module named 'langchain_core'

and section 5 falls back to its 'LLM_DB_PASSWORD not set' note because the notebooks service never receives the password.

Causes. (1) llm/__init__.py eagerly imports HostPool/PoolEmbeddings, so importing llm.config or llm.lineage drags in langchain even though those modules never need it; the notebooks image installs the workspace without the llm extra. (2) The notebook cells import llm.* unguarded, so a missing module is a traceback, not a note — the Task 8 guard only covered the env var. (3) compose.yml passes no LLM_DB_PASSWORD/LLM_PG_HOST to notebooks, so section 5's chunk-per-collection counts can never run there.

Fix. Make llm/__init__ lazy for the pool exports (module __getattr__), so llm.config, llm.lineage, llm.links import with the base deps (section 7b then works replica-only); guard both cells with try/except ImportError → an explanatory note; pass LLM_DB_PASSWORD and LLM_PG_HOST=postgres to the notebooks service so section 5 counts chunks when pgvector is reachable (the container is on the data/storage networks); add the notebooks-container run to the nightly notebook set's expectations. Done when both cells render inside the notebooks container (headless marimo export in-container) and tests cover the import guard.

**Symptom** (notebooks container, marimo://code_families.py cell-14 and the 7b cell): from llm.config import load as _load_llm_cfg File /home/kert/src/llm/__init__.py, line 21: from llm.pool import HostPool File /home/kert/src/llm/pool.py, line 22: from langchain_core.embeddings import Embeddings ModuleNotFoundError: No module named 'langchain_core' and section 5 falls back to its 'LLM_DB_PASSWORD not set' note because the notebooks service never receives the password. **Causes.** (1) `llm/__init__.py` eagerly imports `HostPool`/`PoolEmbeddings`, so importing `llm.config` or `llm.lineage` drags in langchain even though those modules never need it; the notebooks image installs the workspace without the `llm` extra. (2) The notebook cells import `llm.*` unguarded, so a missing module is a traceback, not a note — the Task 8 guard only covered the env var. (3) `compose.yml` passes no `LLM_DB_PASSWORD`/`LLM_PG_HOST` to `notebooks`, so section 5's chunk-per-collection counts can never run there. **Fix.** Make `llm/__init__` lazy for the pool exports (module `__getattr__`), so `llm.config`, `llm.lineage`, `llm.links` import with the base deps (section 7b then works replica-only); guard both cells with try/except ImportError → an explanatory note; pass `LLM_DB_PASSWORD` and `LLM_PG_HOST=postgres` to the notebooks service so section 5 counts chunks when pgvector is reachable (the container is on the `data`/`storage` networks); add the notebooks-container run to the nightly notebook set's expectations. Done when both cells render inside the `notebooks` container (headless `marimo export` in-container) and tests cover the import guard.

kert added this to the P49: Code families as first-class objects — element model, lineage anchors, longitudinal chat milestone 2026-09-11 14:27:26 +00:00

kert added the llm notebooks labels 2026-09-11 14:27:26 +00:00

kert referenced this issue from a commit

2026-09-11 14:31:05 +00:00

fix(llm,notebooks): code_families sections 5/7b run in the notebooks container — lazy pool/sqlalchemy imports in llm, guarded cells, LLM_DB_PASSWORD + psycopg for the notebooks service (refs #720)

kert commented

2026-09-11 14:31:08 +00:00

Fixed on main. (1) llm/init resolves HostPool/PoolEmbeddings lazily and llm.evidence imports sqlalchemy only when a pgvector query runs, so llm.config/llm.links/llm.lineage/llm.evidence import without langchain or sqlalchemy (test: tests/llm/test_init_lazy.py runs a subprocess that blocks those modules). (2) Both notebook cells guard their imports: 7b renders a note instead of a traceback when the llm package cannot be imported; section 5 distinguishes 'client not installed' from 'password not set'. (3) compose.yml passes LLM_DB_PASSWORD and LLM_PG_HOST=postgres to the notebooks service; the notebooks image adds sqlalchemy + psycopg[binary] so section 5 can count chunks. Verified inside the running notebooks container (bind-mounted src): lineage_evidence('history of 99490') builds 57 events with labels, headless export has no traceback. Section 5's chunk counts need the image rebuild + container recreate (docker compose build notebooks && docker compose up -d notebooks); until then it shows its note.

Fixed on main. (1) llm/__init__ resolves HostPool/PoolEmbeddings lazily and llm.evidence imports sqlalchemy only when a pgvector query runs, so llm.config/llm.links/llm.lineage/llm.evidence import without langchain or sqlalchemy (test: tests/llm/test_init_lazy.py runs a subprocess that blocks those modules). (2) Both notebook cells guard their imports: 7b renders a note instead of a traceback when the llm package cannot be imported; section 5 distinguishes 'client not installed' from 'password not set'. (3) compose.yml passes LLM_DB_PASSWORD and LLM_PG_HOST=postgres to the notebooks service; the notebooks image adds sqlalchemy + psycopg[binary] so section 5 can count chunks. Verified inside the running notebooks container (bind-mounted src): lineage_evidence('history of 99490') builds 57 events with labels, headless export has no traceback. Section 5's chunk counts need the image rebuild + container recreate (docker compose build notebooks && docker compose up -d notebooks); until then it shows its note.

kert commented

2026-09-11 14:39:38 +00:00

Verified after the image rebuild + recreate (fhirworx/notebooks:latest, 2026-09-11): inside the notebooks container sqlalchemy 2.0.52 and psycopg 3.3.5 are present, LLM_DB_PASSWORD/LLM_PG_HOST are set, and section 5's pgvector query returns the CCM chunk counts (comments 949, corpus 1,179, rules 428); section 7b builds the timeline (57 events for 99490). Headless export in-container has no traceback. Closing.

kert closed this issue

2026-09-11 14:39:38 +00:00

kert referenced this issue

2026-09-11 20:29:53 +00:00

[nb] code_families.py: exception: No module named 'langchain_core' #719

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: homelab/stack#720