Summary of changes since last commit (a7f5ce7 "Add Stage 1 cross-encoder fast filter for the AI match queue") All changes below are still UNCOMMITTED as of this writing. FEATURES -------- - Legal-form conflict scoring (src/normalize/legal-form.js, new file): After candidate matching, compares legal-entity suffixes (Ltd/LLC/Inc/GmbH/AG/etc.) between source and target names. Boosts confidence when forms agree (synonyms/ abbreviations), rejects `direct`-link candidates when forms genuinely conflict (e.g. GmbH vs AG). Translation-ambiguous forms (LTD/LLC/INC/CORP/PLC/KK/PTY/PVT) are treated as a non-diagnostic cluster to avoid false conflicts on China/Japan translation variance. - tm_xref matching method (src/match/tm-xref.js, new file): Cross-references patent applicant names against already-vetted trademark_holder -> company matches for a new deterministic bridge. - Dotted legal-suffix recognition (src/normalize/names.js): Suffix stripper now recognizes dotted abbreviations (B.V., N.V., A.G., S.p.A., K.K.), previously invisible to both normalization and legal-form scoring. - Ambiguous fuzzy-tie guard for patents (src/pipeline/match-patent.js): dropAmbiguousFuzzyTies() rejects fuzzy matches with multiple similarly-scored candidates; patents dropped from the AI/Ollama review path entirely (raised FUZZY_MIN_RATIO 85->92, all AI imports removed). - Stage1 audit tool (src/pipeline/audit-stage1.js, new file): Re-runs Ollama review over stage1_confirmed rows to validate the cross-encoder classifier's precision on live data; generalized to support both trademark_holder and patent_applicant sources. - Person-name guard in LEI matching (src/match/lei.js): Skips LEI matching for names shaped like personal names (comma-format or trailing-initial) to reduce false positives. BUG FIXES --------- - Legal-form false conflicts: fixed by introducing the ambiguous-cluster concept (was wrongly flagging ~21% of valid matches). - KFT (Hungarian LLC-equivalent) merged into the LTD synonym family -- was wrongly flagged as a conflict against "Ltd". - Dotted-suffix double bug: (1) no dotted regex variants existed at all, (2) even after extraction, dots in the matched token weren't stripped before the family lookup -- both fixed. - Generic-word fuzzy collisions: added wholesale, nutraceuticals, alimentos, bebidas, food/foods etc. to FUZZY_STOPWORDS after they caused cross-source "magnet" false positives. - Fuzzy confidence ceiling: capped legal-form-boosted fuzzy matches below the "settled" auto-skip threshold (0.94) so they still get human/AI review. - ai-queue Phase 2 accidental backlog run: corrected build:ai-queue usage/docs after an unscoped run queued 812K no-match rows instead of just the intended uncertain set. - Unicode EN-SPACE in patent applicant names: worked around via LIKE pattern matching where exact string equality silently failed. - Multiple manually-rejected false-positive matches (person-name collisions, generic-word "magnet" collisions) cleaned up via human_rejected. SINCE THE feature/visualizer BRANCH MERGED INTO master (PR #3, commit fcab835) ------------------------------------------------------------------------------- Six commits landed on top of that merge (all on branch stage1-fast-filter, which starts from master right after the merge; the last one, a7f5ce7, is the "last commit" referenced above and is not yet merged back to master): 1. 1f58bf0 - Added Contacts entity to the visualizer. UI/UX fixes and improvements. Adds a new "Contacts" entity end-to-end: schema column, store helper, collect-db.js ingestion, api-server.js exposure, and visualizer graph.js rendering support (src/db/schema.js, src/db/store.js, src/pipeline/collect-db.js, src/server/api-server.js, visualizer/src/graph.js). 2. 6ce1782 - Pop-ups improvement Reworked the visualizer's node-details popup (visualizer/src/details.js, +148/-.. lines), added a crosshairs-gps icon, and adjusted layout/styling (main.js, style.css, index.html). 3. 5ef673f - UI/UX fixes and improvements. Broader visualizer UI pass: new icons (close-circle, eraser, filter, menu-down, pin/pin-off), reworked filters.js and search.js, and layout/ style updates across index.html, graph.js, main.js, style.css. 4. 15d619f - Add collapsible Stats and Legend panels with icon buttons Stats and Legend panels now collapse to floating icon buttons (chart-arc, map-legend SVGs) stacked bottom-left; clicking expands the panel with a slide animation, both can be open simultaneously. Also added a CHANGELOG.md entry. 5. 4f0df28 - Redirect to login on 401 in client API calls visualizer/src/api.js now redirects to the login page whenever a client API call comes back 401, instead of failing silently/inconsistently. 6. a7f5ce7 - Add Stage 1 cross-encoder fast filter for the AI match queue Adds a fine-tuned cross-encoder (cross-encoder/ms-marco-MiniLM-L-6-v2, ONNX export loaded via @huggingface/transformers) that scores trademark_holder name pairs locally in milliseconds and auto-approves confident ones, skipping the slow Ollama review step entirely for those. - training/: dataset extraction, training, and threshold-calibration scripts (isolated Python venv, never imported by the Node app) - src/match/stage1-classifier.js: loads the ONNX model, exposes stage1Score() - run-ai-queue.js: opt-in via STAGE1_MODE (off/shadow/live, default off) — auto-approve only, no auto-reject (calibration found reject isn't safe for this population); new stage1_confirmed match_method wired through clearAiMatches, migrateSetLinkType, match-tm.js dedup guards, check-legal-forms, export-spreadsheets, and the visualizer legend Shadow- and live-tested against real queue data before merging; full writeup in docs/STAGE1-FAST-FILTER-PLAN.md and STATE_AND_CHANGELOG.md. Everything in the FEATURES/BUG FIXES sections above happened after a7f5ce7, on top of this same stage1-fast-filter branch, and remains uncommitted.