# WIMS Entities Mapper — Current State & Changelog _Last updated: 2026-06-09 (fourth third-party review; 130 FP rows deleted, isPersonNameInitialSuffix guard added to fuzzy.js)_ This document is a snapshot of where the entity-matching pipeline stands today — what data exists, what's matched, how confident we are in it, and how we got here. For the original architecture/plan see [`PLAN.md`](./PLAN.md). --- ## 1. What this project does Bridges the **name-only link** gap in the WIMS dataset: trademark holders and patent applicants are stored as free-text names with no foreign key to the `companies` table (13.7K records). This pipeline produces confidence-scored `entity_matches` rows linking holder/applicant names to company records, using a cascade of deterministic matchers (exact, DBA, LEI, fuzzy) followed by AI adjudication (Ollama, local) for uncertain cases. **Security constraint (always in effect):** a read-only SSH tunnel exists to the production Postgres DB (`wims2`). **No mutation queries against production — ever.** All pipeline state lives in local SQLite at `data/wims.db`. --- ## 2. Local data inventory (`data/wims.db`) | Table | Rows | Notes | |---|---|---| | `companies` | 13,767 | target entities — **not deduplicated**: same-name rows are distinct per-jurisdiction WIMS registrations linked via `parent_company_id` (1,657 rows have one); see [[project-companies-no-dedup]] and §4b | | `trademark_holders` | 1,262,112 | unique local holder names; `country`/`state`/`city` now sourced via the corrected `trademark.office` join (was silently wrong before — see §4b) | | `patent_applicants` | 1,479,367 | unique local applicant names | | `lei_records` | 3,075,680 | GLEIF Level 1 (legal entity reference data) | | `lei_relationships` | 254,335 | GLEIF Level 2 (parent/child relationships) | | `wikidata_companies` | 13,749 | Wikidata enrichment (aliases, sectors) | | `ai_queue` | 11,411 | TM uncertain-match review queue (see §5) | --- ## 3. Match results — current state ### Trademark Holder → Company (14,706 rows / 8,889 valid holders) | Method | Link type | Count | Avg confidence | Notes | |---|---|---|---|---| | `exact` | direct | 5,819 | 1.000 | high-conf, not AI-reviewed | | `ai_confirmed` | direct | 3,473 | 0.819 | | | `ai_confirmed` | subsidiary | 1,347 | 0.810 | | | `ai_rejected` | direct | 2,764 | — | kept for audit | | `ai_rejected` | subsidiary | 1,073 | — | kept for audit | | `dba` | direct | 228 | 0.994 | | | `fuzzy` | direct | 1 | 0.675 | | | `lei` | direct | 1 | 0.828 | | **Valid TM matches:** 10,869 rows / **8,889 unique holders** (0.70% of 1,262,112) _`wikidata` and transliteration methods no longer appear as standalone rows — those original-method matches were in the `conf 0.50–0.94` uncertain range and were updated in-place by the AI queue (122 `wikidata` confirmed, 58 rejected; 59 translit confirmed). The `exact` count (5,819) reflects the false-positive fix re-scan (commit `ea1a44a`): −55 vs. pre-fix run, eliminating numbered-company and CJK personal-name false matches. AI confirmation rate 55.9% (4,864/8,700). Second review (§6c) removed 5 additional pre-fix fuzzy ai_confirmed false positives._ ### Patent Applicant → Company (10,379 rows / 7,952 unique applicants) _Post §6c+§6d+§6e+§6f fixes (Round 4 third-party review applied 2026-06-09)._ | Method | Link type | Count | Avg confidence | Notes | |---|---|---|---|---| | `fuzzy` | direct | 4,466 | 0.774 | −126 vs §6e (person-initial epidemic + cross-sector FP removed) | | `exact` | direct | 2,873 | 0.998 | | | `lei` | direct | 1,902 | 0.883 | | | `lei` | subsidiary | 724 | 0.868 | −1 vs §6e (Chicago Board of Trade LEI anomaly removed) | | `wikidata` | subsidiary | 226 | 0.800 | −13 vs §6e (Merck Canada + Middleton variants removed) | | `dba` | direct | 188 | 0.950 | | **Positive patent matches:** 10,379 (9,429 direct + 950 subsidiary) ### Backups (full snapshots of `entity_matches`, kept for comparison/rollback) - `entity_matches_llama31_backup` — first full AI run (llama3.1), 9,222 confirmed - `entity_matches_mistral_backup` — mistral v1, "same entity?" prompt, 9,355 confirmed / 2,048 rejected - `entity_matches_mistral_v2_backup` — mistral v2, link_type-aware prompt, 10,945 confirmed / 465 rejected (**current production state**) - `entity_matches_pre_lei_fix_backup` — snapshot before the LEI false-positive hardening (commits `3249db8`/`655aa9a`) - `entity_matches_ai_backup` — pre-AI-queue snapshot --- ## 4. The `link_type` feature (newest addition) Every `entity_matches` row now carries a `link_type`: - **`direct`** — same legal entity (name variation, abbreviation, transliteration, rebrand) - **`subsidiary`** — parent/branch/division/holding relationship - **`affiliate`** — reserved, not yet populated This was added so the WIMS site can render weak (indirect/corporate-family) links distinctly from direct identity links in its UI. Implementation summary: - `src/db/schema.js` — `link_type TEXT NOT NULL DEFAULT 'direct'` column + migration - `src/match/lei.js` — reg-number & same-entity paths emit `direct`; parent-walk emits `subsidiary` (LEI confidence is fully deterministic per path: 0.88/0.792 parent-walk, 0.92/0.828 same-entity, 0.95/0.855 reg-number) - `src/match/ai.js` — `aiMatch()` returns `{ score, link_type }`; new prompt explicitly classifies `direct` / `subsidiary` / `none`; `aiScore()` kept as a thin shim over `aiMatch()` - `src/db/store.js` — `updateMatchConfidence`/`clearAiMatches` preserve `original_link_type` in `metadata` for full reversibility; `migrateSetLinkType()` backfills pre-existing LEI rows from confidence bands - `index.js` — `migrate:link-type` command runs the one-time backfill ### Why the v1 → v2 prompt change mattered | Metric | v1 (binary "same entity?") | v2 (link_type-aware) | Δ | |---|---|---|---| | AI confirmed | 9,355 | 10,945 | **+1,590** | | AI rejected | 2,048 | 465 | −1,583 | | Acceptance rate | 82.0% | 95.9% | +13.9 pp | Of 11,402 shared decisions, 9,789 (85.9%) agree between runs and only **15** matches that v1 accepted were rejected by v2 (negligible regression). The **1,598** matches v2 newly confirmed were previously rejected purely because the old prompt forced a binary same-entity framing — the model now correctly classifies them as `subsidiary` instead of discarding them. Breakdown of v2's 1,157 confirmed subsidiary links by original deterministic method: fuzzy 615, lei (parent-walk) 506, exact 20, dba 15, translit_greek 1. --- ## 4b. Location-based direct-link matching (new) `companies` is **not** a deduplicated legal-entity table — same-name rows are distinct **WIMS entity records**, one per jurisdiction/state registration, linked via `parent_company_id` (1,657 rows now have one). Collapsing them would lose real per-jurisdiction granularity (see [[project-companies-no-dedup]]). Instead, when name-matching returns several same-name candidates (the "fan-out" problem — e.g. 37 distinct `NJOY, LLC` rows), `src/match/location.js` now disambiguates by **comparing structured location data**: - `resolveByLocation(candidates, source, companiesById, byParent)` ranks candidates by country+state agreement against the source's location. It only acts when **exactly one** candidate strictly outranks the rest — otherwise it returns the candidate list unchanged, so it can never make a name-match worse. - The winning candidate is emitted as `direct`; its `parent_company_id` and all sibling/child rows in its corporate family are emitted as `subsidiary` (confidence capped at 0.80). - Wired into both `match-tm.js` (full country/state/city — `trademark_item_holder` carries all three) and `match-patent.js` (country only — `patent_applicant` has no state/city). - Validated against the real 37-way `NJOY, LLC` collision: resolved to exactly 1 `direct` (US/North Dakota, conf 1.00) + 44 `subsidiary` links (parent + siblings, conf 0.80). **Bug found & fixed while building this:** `trademark_item_holder.country_id` is an FK to `trademark.office` (filing-office/jurisdiction codes, e.g. `office.code = 'CN'`), **not** `api.countries` as the old joins assumed — they silently produced wrong country codes (e.g. a Shenzhen-based holder stored as `'CC'` Cocos Islands instead of `'CN'`). Fixed in `collect-db.js` and the `pgHolders` fallback in `match-tm.js` (now joins `trademark.office`, selects `o.code`, and also pulls `state`/`city`). A surgical UPDATE-only patch script corrected `country`/`state`/`city` for all 1,262,112 existing `trademark_holders` rows in place — **without** touching `entity_matches` (a full re-collection would have wiped and required rebuilding ~14,900 AI-reviewed matches). Result: 1,064,063 holders now carry a usable `country`, 20,335 carry a `state`. `patent_applicant.country_id` correctly FKs to `api.countries` — no equivalent bug on the patent side. **New schema:** `companies.parent_company_id` / `region_code` / `region_name` (`NULL` = not yet fetched, `''` = fetched but no region — region data comes from the per-company REST endpoint, `api.country_regions` has no SELECT grant over the read-only tunnel) and `trademark_holders.state`. `collect:companies-full` now also backfills region data; of 13,767 companies, 6,280 have a region and 7,487 confirmed to have none. --- ## 5. AI queue pipeline — operating rules **Permanent rule:** the AI queue runs only on **uncertain existing matches** (Phase 1 — confidence 0.50–0.94, not already AI-reviewed). Never queue no-match FTS candidates (Phase 2) unless explicitly requested — set `QUEUE_FTS_MAX_POS=0` to suppress Phase 2 entirely. See [[feedback-ai-queue-uncertain-only]]. Relevant env knobs (`build-ai-queue.js`): - `QUEUE_SKIP_UNCERTAIN=1` — omit Phase 1 - `QUEUE_FTS_MIN_POS` / `QUEUE_FTS_MAX_POS` — bound which FTS candidate positions Phase 2 queues (or skip Phase 2 entirely with `MAX=0`) **Current `ai_queue` state (TM):** fully drained — 1 rejected · 3 errors (stale company refs). After the false-positive fix re-scan (commit `ea1a44a`), only 4 uncertain matches remained; queue was rebuilt and drained in <1 min. The prior full cycle: 8,704 entries → 4,864 matched (55.9%) / 3,836 rejected in 294.5 min (mistral:latest, MIN_SCORE=0.75). --- ## 6. Recent commit history (chronological, oldest → newest) | Date | Commit | Summary | |---|---|---| | 2026-05-28 | `6f28c64` | Initial project scaffold | | 2026-05-30 | `ee13e13` | First matching iteration (exact/fuzzy) done | | 2026-05-30 | `f56e73f` | AI match service + benchmarks | | 2026-05-31 | `564cbf5` | GLEIF + Wikidata enrichment, LEI match step, AI pipeline improvements | | 2026-05-31 | `7a95af2` | Prioritized AI queue (`build:ai-queue` + `run:ai-queue`) | | 2026-06-01 | `ade7f9b` | Second iteration fixes | | 2026-06-01 | `f4898c7` | WIMS DB integration | | 2026-06-02 | `5b286c8` | Fuzzy match improvements | | 2026-06-04 | `76577ec` | Benchmark pipeline, calibrated Mistral v3 prompt, MIN_SCORE → 0.75 | | 2026-06-04 | `2f92d27`, `dc9c9b6`, `1fe8843`, `b9b43e6` | Manual review sample reports + verification agent prompts/results | | 2026-06-04 | `344cc4d` | GLEIF Level 2 relationship import (`collect:lei-rel`) | | 2026-06-04 | `3249db8` | Fix LEI false positives via `lei_relationships` + subset guard | | 2026-06-04 | `655aa9a` | Harden LEI match: separate parent-walk similarity threshold | | 2026-06-04 | `2df9164`, `fe91880` | `QUEUE_SKIP_UNCERTAIN`, `QUEUE_FTS_MIN_POS`/`MAX_POS` queue-shaping knobs | | 2026-06-05 | `2c2336d` | Skip Phase 2 early when no FTS positions selected | | 2026-06-05 | `fb379c5` | Preserve original method/confidence/link_type through AI updates (`clearAiMatches` reversibility) | | 2026-06-05 | `a50d771` | **`link_type` feature**: `direct` vs `subsidiary` classification across LEI, AI, and storage layers | | 2026-06-08 | `13d1251` | Location-based direct-link matching (`src/match/location.js`), `trademark_item_holder.country_id` join-bug fix, `companies.parent_company_id`/`region_code`/`region_name` + `trademark_holders.state` columns — see §4b | | 2026-06-08 | `5a2f312` | Wikidata parent/subsidiary relations collection (`wikidata_relationships`, `relations_checked` two-pass sync) — backlog #1 | | 2026-06-08 | `0b3fe10` | Wire Wikidata corporate-ownership graph into matching (`wikidataParentWalk`/`buildWikidataIndex` in `lei.js`, new `wikidata` match method) | | 2026-06-08 | `5c10673` | Fix `wikidataParentWalk` false bridges via generic sector words (generalized `distinctiveTokensSubsetOf`/`GENERIC_CORPORATE_TOKENS`, `WIKIDATA_SINGLE_TOKEN_MIN_SIMILARITY=95`) | | 2026-06-08 | `9cfafe0` | Document live wikidata match validation results and Wikidata data-quality caveat | | 2026-06-08 | `504fc8c` | `aiResolvedIds` guard in `match-tm.js`: skip holders already carrying a final `ai_confirmed`/`ai_rejected` decision during deterministic re-scans (prevents duplicate rows) | | 2026-06-08 | `d204edd` | Full from-scratch re-scan (all matches cleared, 16-worker PM2, 1,262,112 holders in 22.3 min → 12,113 matches); rebuild AI queue (8,704 uncertain entries); AI queue run completed (294.5 min, 55.9% confirm rate) | | 2026-06-09 | `ea1a44a` | Fix three false-positive sources found in AI queue spotcheck — see §6a | | 2026-06-09 | `df2da1b` | Fix four false-positive sources identified by third-party AI review — see §6b | | 2026-06-09 | `04bbf52` | Fix LEI R&D cluster (hasSharedDistinctiveToken guard), personal-name comma guard, 347 rows deleted — see §6c | | 2026-06-09 | `7124d96` | Fix `&`→`amp` fake token: add `'amp'` to `FUZZY_STOPWORDS` so HTML-entity artifact never acts as distinctive brand token — see §6d | | 2026-06-09 | `930b6e1` | Round 3 third-party review: 90 FP rows deleted, 5 wikidata_relationships removed, geographic/semiconductor tokens added to GENERIC_CORPORATE_TOKENS — see §6e | | 2026-06-09 | `b2b82f2` | Round 4 third-party review: 130 FP rows deleted, isPersonNameInitialSuffix guard added to fuzzy.js — see §6f | ### §6a. False-positive fixes (commit `ea1a44a`, 2026-06-09) Three root causes were identified by spotchecking the AI queue results: **1. Numbered Canadian company guard (`src/match/fuzzy.js`)** Registration-number company names (e.g. `82532 Canada Ltd`, `9385-4495 Quebec Inc.`) were producing fuzzy false positives against other numbered companies because `token_sort_ratio` on "Canada Inc." / "Quebec Inc." is high regardless of the registration numbers. `fuzzyMatch()` now returns `[]` immediately when the source name matches `^\d[\d-]+ (Canada|Quebec|Ontario|...)` — these can only match via exact name comparison. The exported `isNumberedCompany()` predicate is available for other callers. **2. CJK/Hangul personal name guard (`src/match/transliterate-match.js`)** Short CJK/Hangul names (≤4 non-space characters, ≤3 transliterated words, no corporate-entity tokens) are almost always personal names — `李明`, `寺川 祐一`, `하만` — but were being phonetically matched to companies with similar-sounding names (Liming Technology, HAMAN RESOURCES, etc.). `looksLikePersonalName()` now short-circuits `transliterateMatch()` for these. Longer CJK names and katakana company names (e.g. ピーティー サトリア アグロ インダストリ) pass through unchanged. **3. `ai_score` propagated to `entity_matches.metadata`** Mistral's numeric confidence score was only stored in `ai_queue.ai_score` — not copied to `entity_matches.metadata`, losing auditability. `updateMatchConfidence()` now accepts an optional `aiScore` parameter written as `metadata.ai_score`. Both the in-place `uncertain` update path and the `no_match` new-insert path in `run-ai-queue.js` now save the score. Takes effect for the next AI queue run. ### §6b. Third-party AI review fixes (commit `df2da1b`, 2026-06-09) 38 matched pairs (TM ↔ Company) were sent to a third-party AI agent for quality review. Key findings led to four code fixes: **1. Wikidata link_type preserved through AI review (`store.js`)** AI was reclassifying parent-walk `subsidiary` links as `direct` when the names looked like the same entity (e.g. HEINEKEN ESPANA → Heineken N.V.). Fixed in `updateMatchConfidence()`: wikidata-origin rows now ignore AI's link_type and always keep the original `subsidiary`. 68 existing rows in `entity_matches` corrected in-place. **2. Single-token fuzzy matches require near-exact ratio (`fuzzy.js`)** After applying stopwords, sources that reduce to a single distinctive token (e.g. "delight", "yixi") matched phonetically similar but unrelated company names at the 85-point threshold. `effectiveMinRatio` is now raised to 95 for single-token sources — blocks "delight"→"delightex", "yixi"→"yunxi". **3. Malaysian corporate suffixes added to FUZZY_STOPWORDS (`names.js`)** `sdn` and `bhd` (Sdn Bhd = Malaysian private limited) were acting as shared tokens inflating similarity between unrelated companies. **4. Legacy CJK personal name `ai_confirmed` rows removed (data-only)** 39 holders with short CJK personal names (李明, 寺川 祐一, 하만, etc.) had been confirmed by AI before the `transliterateMatch` personal-name guard was added. One-time cleanup removed all 39; the katakana company name ピーティー サトリア was correctly preserved. _Note: short DBA acronym guard (SIC, TMS, MDM flagged as UNCERTAIN) was not implemented — a length guard on DBA aliases would break correct JT→Japan Tobacco matches (confirmed correct by the same reviewer). Those cases remain AI-reviewed at conf 0.855–0.90 and subject to the standard review queue._ ### §6c. Second third-party AI review fixes (2026-06-09) 41 matched pairs (patent + TM) were sent to a second AI agent for quality review. The reviewer identified two systematic patterns and confirmed the wikidata link_type corrections from §6b. Total: **347 rows deleted** across patent and TM sides. **1. LEI parent-walk R&D cluster (`lei.js`): 295 rows deleted** The most critical finding: **226 patent applicants were falsely linked to Nestlé S.A.** and **69 to Kopran Limited**, both via the GLEIF parent-walk. Root cause: each company has a GLEIF-registered subsidiary with "Research & Development" (or "Research Laboratories") in its name. Patent applicants like "Deka Research & Development Corp" (Dean Kamen's medical-device R&D firm) and "Yeda Research & Development Co Ltd" (Weizmann Institute's tech-transfer arm) shared those generic tokens with the Nestlé GLEIF subsidiary, gave a similarity score ≥72%, and were walked up to Nestlé S.A. as the "parent company." Two-part fix in `lei.js`: - Added `"development"`, `"centre"`, `"center"` to `GENERIC_CORPORATE_TOKENS` (these appear in subsidiary names of many large companies and carry no entity-identity signal). - Added `hasSharedDistinctiveToken(norm1, norm2)` function and wired it as a guard into the parent-walk path: source and GLEIF candidate must share at least one token that is not in `GENERIC_CORPORATE_TOKENS` (e.g. "silicon" for Silicon Labs Finland → Korea sibling-bridge, "tobacco" for BAT subsidiaries). Without a shared brand token, the parent-walk is rejected. Confirmed correct matches (BAT 131, Sanofi 90, TE Connectivity 80) all have a shared distinctive brand token and are unaffected. **2. Personal-name "Lastname, Firstname" guard (`fuzzy.js`): 9 rows deleted** "Morris, Phillip" (EPO patent inventor format for Phillip Morris the person) matched to Philip Morris International at fuzzy confidence 0.90. The name appeared 9 times as both "Morris, Phillip" and "Morris, Phillip L". Added `isPersonNameCommaFormat()` to `fuzzy.js`: rejects fuzzy matching for names with exactly one comma where the after-comma part is ≤2 words and neither part contains a corporate suffix token. Applies to both TM and patent pipelines. The guard does not fire for "Philip Morris, Inc." (has "inc" corp token) or "Oglesby & Butler Research, Ltd" (has "ltd"). **3. Wikidata source-data false positives: 23 rows deleted** Three Wikidata corporate-relationship claims are wrong/outdated: - Eltek SpA (Italian auto-parts firm) ≠ Delta Electronics' subsidiary Eltek AS (Norwegian power supply company) — 5 rows across multiple Eltek variants. - Merck Frosst Canada (Merck & Co. / US Merck subsidiary) ≠ Merck KGaA (unrelated German company; the two Mercks have different histories in the Americas vs Europe) — 6 rows. - Wilkinson Sword → Swedish Match: ownership was brief in the 1980s and is now outdated (Wilkinson Sword is owned by Edgewell Personal Care) — 12 rows. All 23 rows deleted from `entity_matches`. The underlying Wikidata claims remain in `wikidata_relationships` — they will fire again on next full re-scan, but the TM side already had these rejected by the AI queue (`ai_rejected`), confirming the AI would catch them if an AI queue run were done for patents. **4. Residual pre-fix fuzzy ai_confirmed false positives (TM): 5 rows deleted** The §6b code fixes prevent NEW matches of these types, but `ai_confirmed` rows from the original AI queue run (before fixes) survived the fix re-scan. 5 rows confirmed as false positives by the reviewer: "Ceed Vitality Group" (over-weighted "vitality"), "DNA Nutraceuticals" → 2 targets (generic nutraceuticals token), "Lin Luo" (personal name mapped to a company), "PT. ZUNA GROUP INDONESIA" → Pt You Indonesia (PT/Indonesia tokens inflated similarity). **5. Patent geographic false positives: 15 rows deleted** - Chongqing China Tobacco (Chongqing province): matched to 4 other provincial tobacco companies (Yunnan, Sichuan, Hunan, Shandong) because `yunnan`/`chongqing`/`sichuan`/ `hunan`/`shandong` are all in `FUZZY_STOPWORDS`, leaving only "china tobacco" as shared signal → high similarity between all provincial bureaus. Deleted 12 cross-province rows (kept 3 Chongqing→CNTC matches since CNTC is the legitimate national parent). - Shenzhen Woody Vapes Tech: matched to "Dongguan Woody Vapes Technology" (wrong city) and "Voopoo International" (unrelated). 3 rows deleted; the correct Shenzhen→Shenzhen Woody Vapes and Woody Vapes Inc→Woody Vapes Inc matches preserved. ### §6d. `&`/`amp` HTML entity artifact fix (commit `7124d96`, 2026-06-09) After applying the `hasSharedDistinctiveToken` guard (§6c), a match:patents re-scan revealed **203 Nestlé false positives still present**. Investigation showed they were entering via a different path than the one §6c fixed. **Root cause:** Both `patent_applicants.name` (sourced from WIMS) and `lei_records.legal_name` (GLEIF XML) store HTML entities — `&` appears as `&`. `normalizeName()` strips punctuation with `/[^\w\s]/g`, which turns `&` into the space-separated token `amp` (3 chars, not in `GENERIC_CORPORATE_TOKENS` at the time). Any two entities with `&` in their names therefore shared the token `"amp"`, which satisfied `hasSharedDistinctiveToken` and passed the guard as if they had a genuine shared brand token. Example trace: - Patent applicant: `"Yeda Research & Development Co Ltd"` → norm: `"amp development research yeda"` - GLEIF entity: `"GALDERMA RESEARCH & DEVELOPMENT"` → norm: `"amp development galderma research"` (Galderma = Nestlé subsidiary in GLEIF) - `hasSharedDistinctiveToken("amp development research yeda", "amp development galderma research")` → **TRUE** (shared: `"amp"`) — gate passed, Yeda falsely linked to Nestlé **Fix:** Added `'amp'` to `FUZZY_STOPWORDS` in `src/normalize/names.js`. Since `GENERIC_CORPORATE_TOKENS` is derived from `FUZZY_STOPWORDS`, `"amp"` now propagates to both the fuzzy-scoring stopword filter and the distinctive-token guard, neutralizing the artifact everywhere. Verified correct behavior after fix: - `hasSharedDistinctiveToken("amp development research yeda", "amp development galderma research")` → **FALSE** ✓ - `hasSharedDistinctiveToken("finland laboratories silicon", "korea laboratories silicon")` → **TRUE** ✓ (Silicon Labs sibling-bridge preserved — `"silicon"` is a real brand token) The 203 Nestlé false positives that persisted after §6c were cleared by deleting the remaining Nestlé/Kopran clusters before the re-scan (those rows were already covered by the §6c data cleanup). The third match:patents re-scan with both fixes confirmed 8,196 new matches (vs 8,390 in the first re-scan before the amp fix). ### §6e. Third third-party AI review (Round 3, 2026-06-09) 49 matched pairs across 13 groups were sent to a third-party AI reviewer. All 49 returned **FALSE_POSITIVE**. Total: **90 rows deleted** from `entity_matches`; **5 records deleted** from `wikidata_relationships`; **1 code fix** in `lei.js`. **Group A: Known Wikidata re-appearing clusters (24 rows deleted + 3 wikidata_relationships deleted)** The three Wikidata false positives documented in §6c as "re-appearing on each fresh re-scan" are now permanently resolved by deleting their source `wikidata_relationships` records: - **A1 Eltek SpA/Srl → Delta Electronics (4 rows):** Wikidata conflated Italian Eltek SpA (automotive circuit boards) with Norwegian Eltek AS (power supplies, genuine Delta subsidiary). `(Q5367793 → Q5254620)` deleted. - **A2 Merck Frosst Canada → Merck KGaA (5 rows):** Classic disambiguation failure — Merck Frosst is a Merck & Co. (US) subsidiary; matching via GLEIF "Merck (Canada)" entry walked up to Merck KGaA (Germany), an unrelated competitor. `entity_matches` rows deleted; `wikidata_relationships` kept (Merck KGaA's Canadian sub is a legitimate entity). - **A3 Wilkinson Sword → Swedish Match (15 rows):** Stale 1980s ownership. Wilkinson Sword has been part of P&G/Edgewell since 1993. `(Q653630 → Q52855)` deleted — eliminates all 15 name variants on future re-scans. **Group B: New Wikidata false-positive clusters (56 rows deleted + 2 wikidata_relationships deleted)** - **B1 TSMC → ON Semiconductor (30 rows):** TSMC is the world's largest independent foundry (NYSE: TSM), completely unrelated to ON Semiconductor. Root cause: "taiwan" and "semiconductor" were both distinctive tokens shared between TSMC variants and "ON Semiconductor (Taiwan)" (Q30343778). Fixed by code (see code fix below). Entity_matches rows deleted. - **B2 Renesas Northern Japan → ON Semiconductor (1 row):** Same root cause as B1 — "japan" and "semiconductor" bridged Renesas to "ON Semiconductor (Japan)" (Q30343779). Fixed by same code fix. - **B3 Orion Corp/Industries → Nestlé (5 rows):** Wikidata has "Orion" (Q12042897) as a Nestlé subsidiary — this is the Czech Orion chocolate brand. Finnish pharma Orion Corporation and US aerospace Orion Industries have no Nestlé connection. `(Q12042897 → Q160746)` deleted. - **B4 ETA SA → HP Tronic (2 rows):** ETA SA is the Swatch Group's Swiss watch movement manufacturer; HP Tronic is a Czech electronics company. No connection. `(Q11866675 → Q54977059)` deleted. - **B5 Middleton person names → Altria (3 rows):** "Middleton John C us", "Middleton John L us", "Middleton Nigel John gb" are patent inventor names, not the John Middleton Inc. cigar company (a real Altria subsidiary). Entity_matches rows deleted; underlying relationship kept. - **B6 Knight person names → Unilever (8 rows):** "Knight John gb", "Knight, John B, Jr" etc. are individual inventors, not the historical John Knight Ltd soap company. Entity_matches rows deleted; underlying relationship kept. - **B7 Alcon → Nestlé (2 rows):** Historically accurate (Nestlé owned Alcon 1977–2010) but ownership ended 15 years ago; Alcon is now independent (NYSE: ALC). `(Q684825 → Q160746)` deleted. **Group C: LEI subsidiary person-name and geographic false positives (6 rows deleted)** - **C1 South Africa cluster → Nestlé (5 rows):** "Behr South Africa", "Plessey South Africa", "Leer South Africa", "Gunson Seeds South Africa", "Noel Lab South Africa" — all matched to Nestlé via GLEIF LEI parent-walk because "south" and "africa" were treated as distinctive brand tokens shared with Nestlé's South African GLEIF subsidiary. Fixed by code (see below). - **C2 Mcneil Tom → Kenvue (1 row):** Person name (Tom McNeil) falsely matched to McNeil Consumer Healthcare (Kenvue). Deleted. **Group D: Suspicious fuzzy direct matches (10 rows deleted)** 10 conf-0.75 fuzzy pairs spanning different geographies, product sectors, and person names: Stout Medical/Sotsu Medical, Fisher Scientific/Inter Scientific, Kirin Machinery/Shanghai Kuixing, Rush Benjamin M/Dr. Benjamin Rush (×2), Sensor Diagnostics/Siemens Healthcare Diagnostics, Fuchs Dan/Shenzhen Fuchs, Hong Jiang/Chang Jiang Electronics, HK Medical Tech/KOR Medical, Nitta Belting/Nitta Gelatin. All deleted. **Code fix: `GENERIC_CORPORATE_TOKENS` extended (`lei.js`)** Root cause of B1/B2 (TSMC/Renesas→ON Semi) and C1 (South Africa→Nestlé): several geographic and industry terms were not in `GENERIC_CORPORATE_TOKENS`, so they acted as "distinctive" brand tokens that falsely bridged unrelated companies sharing a regional presence or sector. Added to `GENERIC_CORPORATE_TOKENS`: - **Geographic directions:** `'south'`, `'north'`, `'east'`, `'west'`, `'africa'` → blocks "South Africa" bridging (C1) and compass-direction subsidiary names - **Country names in subsidiary labels:** `'taiwan'`, `'japan'`, `'korea'`, `'india'`, `'europe'`, `'pacific'` → prevents "ON Semiconductor (Taiwan)" bridging TSMC (B1) and similar country-suffix subsidiaries - **Industry generic:** `'semiconductor'`, `'semiconductors'` → as generic as `'pharmaceutical'` within the sector; shared by thousands of unrelated companies After the fix: "ON Semiconductor (Taiwan)" (Q30343778) has zero distinctive tokens → `wikidataParentWalk` skips it entirely. Same for "ON Semiconductor (Japan)" (Q30343779). ### §6f. Fourth third-party AI review (Round 4, 2026-06-09) 51 matched pairs across 4 groups were sent to a fourth third-party AI reviewer. All confirmed **FALSE_POSITIVE** except two disambiguation rows (CORRECT). Total: **130 rows deleted** from `entity_matches`; **1 code fix** in `fuzzy.js`. **Group A: Single-initial person-name epidemic (100 rows deleted)** Patent inventors filed in "Surname X" or "Surname Firstname X" format (e.g. "Atkins D", "Garrison John M", "Nelson J") were being fuzzy-matched to eponymous companies that share the same surname. The existing `isPersonNameCommaFormat` guard (added in §6c) catches "Lastname, Firstname" comma format but missed these initial-only and multi-word no-comma variants. ~100 rows deleted across 25+ distinct patterns. **New guard `isPersonNameInitialSuffix(name)` added to `fuzzy.js`:** - Returns `true` (skip fuzzy) when the last token is a single uppercase letter AND total token count is 2–4 AND no preceding token is in `FUZZY_STOPWORDS` or `PERSON_NAME_CORP_TOKENS`. - Safeguards: two consecutive single-letter tokens at the end → corporate form abbreviation (A G = AG, B V = BV) → not flagged. A preceding corporate/industry token in `FUZZY_STOPWORDS` (e.g. "electronics", "shenzhen", "manufacturing") → name is a truncated company name, not a person → not flagged. **Group B: Pre-Round-3-fix Shenzhen semiconductor multi-match (15 rows deleted)** Three Shenzhen-based companies each matched 3–4 completely unrelated semiconductor firms simultaneously, proving the match was driven by the now-blocked generic tokens ("semiconductor", "shenzhen") before they were added to `GENERIC_CORPORATE_TOKENS` in Round 3. Rows were created before the fix and not cleared by the re-scan. Representative: - "Shenzhen Kia Semiconductor Tech" → ON Semi + Dialog + ITM + Alpha & Omega (4 targets) - "Shenzhen Tete Semiconductor" → Torex + Semtech ×3 (4 targets) **Group C: Cross-sector and geography mismatches (12 rows deleted)** - Chicago Board of Trade → Colin Mear Engineering Ltd. (1 row, LEI path anomaly) - Oisin Biotechnologies Inc (US longevity) → Shenzhen Oplus Biotechnologies (CN optics) - AO Medical Products AB (Swedish orthopedic) → KOR Medical LLC (US sports medicine) - Wuhan Phoenix Chem / Baf Aktiengesellschaft / Mc Medical Kk — chemistry and pharma name collisions - Green Garden Inc / Winchester Dev Associates / Spink Benjamin — geography + person-name FP **Group D: Disambiguation (3 rows deleted, 3 kept)** - **Merck Canada Inc → Merck KGaA** (3 rows deleted): "Merck Canada Inc." is the subsidiary of Merck & Co. (US Merck/MSD); the German Merck KGaA's Canadian entity operated as "EMD Inc." — same Merck disambiguation as Round 3 A2. - **Panasonic IP Management Corporation → Panasonic Corporation** (2 rows kept, CORRECT): Panasonic IP Management is a real wholly-owned subsidiary established to file and manage the parent company's patent portfolio. - **Jiangsu Highstar Battery Mfg Co Ltd → Jiangsu Highstar Battery Manufacturing Co., Ltd.** (1 row kept, CORRECT): "Mfg" is a standard abbreviation for "Manufacturing" — same company. --- ## 7. Backlog (not yet started) 1. ~~Extend `collect-wikidata.js` to fetch parent/subsidiary relations (Wikidata properties P749, P355)~~ — **done**: new `wikidata_relationships` table (child_qid→parent_qid, mirrors `lei_relationships`), populated for free from claims already being fetched (P749 "parent organization" / P355 "subsidiary", normalized into one direction). A `relations_checked` sentinel on `wikidata_companies` drives a resumable two-pass sync: (a) backfill claims for companies matched before this feature existed, (b) resolve the *other end* of each edge (usually an entity we never searched for) by fetching its label and exact-matching it against the company index. First run: 852 relationship edges collected, 21 resolved into genuine company↔company corporate-ownership links (e.g. Juul→Altria, Tropicana Products→PepsiCo, Wrigley Company→Mars). **Now wired into matching**: `wikidataParentWalk`/`buildWikidataIndex` in `lei.js` mirror the GLEIF parent-walk — when a source name resembles a Wikidata-known entity (`token_sort_ratio ≥ WIKIDATA_MIN_PARENT_WALK_SIMILARITY=75`) and isn't already a likely direct match (`< WIKIDATA_HIGH_SIMILARITY=85` for entities we track as companies — avoids preempting a more-precise fuzzy hit), walk up `wikidata_relationships` to the entity's parent QID and, if that parent was resolved to a `company_id` during collection, emit a flat-confidence (`WIKIDATA_PARENT_CONFIDENCE=0.80`) `subsidiary` link. Runs right after `leiMatch` (before fuzzy) in both `match-tm.js` and `match-patent.js`. Confidence is flat (not `exactMatch`-scaled like LEI's) because the parent end was already deterministically resolved during collection. **Live validation** (150K unmatched holders, deterministic-only cascade) surfaced and fixed two precision bugs unique to brute-force scanning the small (776-entry) Wikidata index without GLEIF's FTS prefilter: (a) alias-based bridging via short generic words (e.g. "Wrigley"), and (b) canonical-label bridging via generic *sector* words — "Eybna Technologies"/"Tribe Breweries"/"Endo Biologics" all matched unrelated "X Technologies"/"X Breweries"/"X Biologics" entities at sim 76-82 sharing only the sector word, then walked to *that* entity's parent (e.g. "Two Apple GmbH" → Apple Inc, "American Airlines" → a Shenzhen shell co). Fixed via a generalized distinctive-token-subset guard (`distinctiveTokensSubsetOf`/`GENERIC_CORPORATE_TOKENS`, shared with `leiMatch`) plus `WIKIDATA_SINGLE_TOKEN_MIN_SIMILARITY=95` (a lone distinctive token — e.g. "breweries" alone — is trusted only when the overall match is essentially exact). Re-run produced **46 genuine wikidata matches** persisted to `entity_matches` (e.g. Stiefel Laboratories→GSK, Mylan→Viatris, Life Technologies→Thermo Fisher, WhatsApp→Meta, Amylin/Acerta Pharma→AstraZeneca, Altadis→Imperial Brands, Sandoz→Novartis). **Caveat observed**: a few resolved links reflect what looks like *wrong/outdated Wikidata claims* rather than matcher errors — e.g. Wikidata's P749 claims list "Ferrara Candy Company"→Procter & Gamble and "Sunny Delight Beverages"→PepsiCo, neither of which matches known ownership history; the matcher's identification was exact (sim=100) in both cases, so this is a source-data quality issue inherent to community-edited Wikidata, not something the matcher can detect — the flat 0.80 confidence + `subsidiary` link_type intentionally route these to the review queue rather than auto-accepting 2. Load GLEIF Level 2 relationship file into `lei_relationships` *(done — see `344cc4d`; remaining: broaden coverage/refresh)* 3. Add a registration-number pre-match step ahead of fuzzy/AI queue 4. Integrate Companies House API for GB companies 5. Extract US subsidiary lists from SEC EDGAR filings 6. Add OpenCorporates enrichment for global registration data 7. ~~Fix ~1,741 duplicate rows in `companies`~~ — **redirected, not a bug**: these are legitimate distinct per-jurisdiction WIMS entity records, not dedup artifacts (see [[project-companies-no-dedup]]). Resolved instead via location-based direct-link disambiguation — see §4b --- ## 8. Reference - **SCP / file server:** `arkotik.dev:/sites/files.arkotik.dev/www/wims-entities-mapper/` — see [[reference-scp]] - **Production DB tunnel:** `127.0.0.1:5432`, db `wims2`, user `db_graph_analyzer` — **read-only, never mutate** - **Pipeline architecture & schema:** [`PLAN.md`](./PLAN.md)