Headline numbers
Deterministic pass (match:tm) — before vs after
entity_matches for trademark_holder was fully cleared (14,193 rows, plus
the fully-dependent ai_queue and review_queue) and rebuilt from scratch via
DISABLE_AI=1 node index.js match:tm with MATCH_CONCURRENCY=16 under PM2.
Completed in 22.4 minutes for all 1,262,112 locally-synced holders — 11,565 new deterministic
matches (0 via the in-pipeline AI fallback, which stayed disabled as intended).
Method breakdown
| Method | Before | After | Δ | Avg confidence (before → after) |
|---|---|---|---|---|
| exact | 4,362 | 4,449 | +87 | 1.000 → 1.000 |
| dba | 173 | 174 | +1 | 0.998 → 0.998 |
| fuzzy | 13 | 18 | +5 | 0.992 → 0.982 |
| lei | 6 | 6 | 0 | 0.960 → 0.960 |
| ai_confirmed | 7,660 | 1,545 | −6,115 | 0.812 → 0.807 |
| stage1_confirmed | 62 (early tests) | 4,188 | +4,126 | 0.997 → 0.998 |
| ai_rejected (kept for audit) | 1,917 | 3,795 | +1,878 | 0.085 → 0.510 |
| Total rows | 14,193 | 14,175 | −18 | 0.775 → 0.847 |
The ai_confirmed → stage1_confirmed shift is the whole point of this exercise:
the same population of holders that used to require a full Ollama call to confirm now mostly gets
confirmed by the local Stage 1 gate instead, at higher and more consistent confidence (0.99+ vs
Ollama's formula-capped 0.85 ceiling).
ai_rejected's average confidence jumped from 0.085 to 0.510: this is a direct,
measurable effect of the parser fix in §4.3. Before the fix, any pair where mistral expressed a
"leaning yes but genuinely torn" response (a numeric match value instead of a strict
boolean) failed to parse at all and was silently discarded as an error after 3 retries —
it never became an ai_rejected row. After the fix, those same borderline pairs parse
correctly and land in ai_rejected with their real, non-trivial confidence value (still
below the 0.75 acceptance bar, but no longer thrown away as noise). The population got more complete
and more honest at the same time.
Structural (link_type) shift
subsidiary classifications nearly tripled (1,257 → 3,745). Most likely explanation: the
"before" state was an accumulation of many incremental rounds — parent/subsidiary detection logic
(location-based fan-out, LEI/Wikidata parent-walk) was added at different points over the project's
history and only partially back-applied to older matches. A single from-scratch pass applies all of
today's detection logic uniformly to the whole population in one shot. Not independently re-verified
row-by-row in this session — flagged as the leading hypothesis, not a confirmed root cause.
Match-count delta, explained
Raw distinct-holder count looks like a small regression (11,565 after vs 11,714 before, −149),
but 86 of those "before" matches came from holders with source_id > 1,308,430 — beyond
the local trademark_holders snapshot's range, reachable only via the single-threaded
path's live-Postgres streaming (used in an earlier catch-up run this session, see §5). The 16-worker
path used for this rescan only ever shards the local snapshot and structurally cannot reach those —
see §5's known-limitation note. Restricting "before" to the same local scope gives 11,628, so the
real like-for-like delta is −63 (−0.54%) — within the range of ordinary variation
from re-running string-matching/candidate-ranking logic from scratch, not a regression signal.
AI review pipeline — throughput progression
9,528 holders landed in the uncertain confidence band (0.50-0.94) and were queued for Stage 1 +
Ollama review (build:ai-queue, Phase 1 only — the 989,841-row no_match
backlog was never touched, per existing policy). Processed via the ai-queue-mistral PM2
app with STAGE1_MODE=live, across four invocations as fixes were applied mid-run.
Final tally
| Outcome | Count | % of processed |
|---|---|---|
| Stage 1 auto-approved (Ollama skipped) | 4,188 | 43.96% |
| Ollama-confirmed | 1,545 | 16.22% |
| Ollama-rejected | 3,795 | 39.83% |
| Total resolved | 9,528 | 100% |
Ollama itself never auto-rejects a Stage-1-approved pair and Stage 1 never auto-rejects at all (by calibrated design, see the Stage 1 plan doc) — every rejection above is a genuine Ollama decision on a pair Stage 1 wasn't confident enough to skip.
Wall-clock time
Total across all four invocations (including the three mid-run pauses for diagnosis/fixes): ~171.5 minutes for 9,528 items — despite the interruptions, faster in total than the 281.6 minutes the historical baseline took for a smaller (8,369-item) batch with no Stage 1 involved at all.
Issues found and fixed this session
Bug match:tm worker process doesn't exit
The 16-worker match:tm run finished its logical work (printed "TM→Company done") but
the underlying Node process stayed alive for 29+ minutes afterward — PM2 reported it as
online indefinitely. Root cause not chased down (didn't block progress, just required a
manual pm2 stop); worth investigating if this pattern recurs, since it makes PM2's own
status an unreliable completion signal for this app.
Performance Fixed Ollama partial CPU offload from oversized context window
AI queue throughput was running at ~20-21/min, roughly half of the historical baseline. Diagnosis:
ollama ps showed 12.2% of the model running on CPU (6.71GB of 7.64GB in VRAM),
and the ollama runner process was pegged at 408% CPU. Root cause: Ollama's default
4096-token context allocates a KV cache far larger than these prompts need (~1,350 tokens measured
worst-case) — on an 8GB card with the desktop compositor also holding VRAM, that oversized allocation
didn't leave room for the full model.
Fix: added an explicit num_ctx option (default 2048, configurable via
OLLAMA_NUM_CTX) to the Ollama chat request in src/match/ai.js.
Result: 0% CPU offload after the fix (confirmed via ollama ps), runner CPU
dropped to 99%, throughput rose to ~28-32/min immediately.
Bug Fixed Numeric match value rejected as unparseable
A background ~1-3% error rate persisted even after the num_ctx fix. Added structured diagnostic
logging (raw response capture on parse failure) and caught it directly: mistral was consistently
returning "match": 0.7 — a graded number — instead of the strict boolean the schema
requires, specifically on pairs it found genuinely borderline. The parser's fallback regex also
only matched literal true/false, so these pairs failed both the primary
and fallback parse paths and were thrown away as Unexpected response shape.
Since generation runs at temperature: 0 (deterministic), retrying changed nothing —
the same malformed response repeated byte-for-byte across all 3 attempts, so every one of these
was guaranteed to burn 3 wasted Ollama calls and land in error.
Fix: parseMatchResponse now coerces a numeric match to boolean
(≥ 0.5) in both the primary JSON path and the regex fallback, in
src/match/ai.js. Verified against the live model against the exact failing pairs before
rolling out.
Result: 88 previously-errored items reset and reprocessed successfully; 0 new errors in the remainder of the run (7,468 items, 0 errored / 0 retrying throughout).
Added Retrying/errored visibility in live progress output
run-ai-queue.js's progress line only ever showed processed
("done") and terminal outcomes — items hitting a transient failure (timeout, unparseable response)
silently consumed a dequeued batch slot without incrementing any visible counter, which is why
"done" appeared to increment by 15-19 instead of the full batch size of 20. Added explicit
errored and retrying counters to both the live progress line and the
final summary so this is visible going forward instead of requiring a DB investigation each time.
Known limitations & scope notes
match:tm path is local-DB-only. Unlike the single-threaded path,
matchWithWorkers() shards purely across the local trademark_holders table's
existing ID range — it has no logic to stream new holders from the live WIMS Postgres source. This
rescan covered all 1,262,112 locally-synced holders, but not the (at least) 8,076 holders known to
exist live but outside the local snapshot (found via the single-threaded path earlier this session).
Not fixed in this session — flagged as a real gap between the two code paths.
no_match backlog (989,841 rows) was never touched. Per existing policy, only
the "uncertain" reason queue is processed. Stage 1 calibration was also never done for that
population — its negatives never passed a similarity screen, so they're structurally different from
(and may be easier to auto-reject than) the uncertain queue's negatives. Separate future effort.
STAGE1_MODE defaults to off. None of this changes standing
configuration — every live run in this report was an explicit, scoped, user-authorized invocation.
Timeline
wims-pre-fullrescan-20260701-182942.db), integrity-checked.entity_matches/ai_queue/review_queue cleared for trademark_holder (14,193 / 1,004,005 / 30 rows).match:tm run via PM2, 16 workers, AI disabled. 22.4 min, 11,565 new matches, 1,262,112 processed.build:ai-queue (Phase 1 only) — 9,528 uncertain matches queued.STAGE1_MODE=live. Throughput ~20-21/min — flagged as abnormally slow.num_ctx=2048. Restarted.errored/retrying progress counters. Restarted."match": 0.7 parser bug directly. Fix implemented and verified live. 77 stale errors reset. Restarted.