Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/design/decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -370,6 +370,12 @@ Excluded (SUFFIX_ACRONYMS / SUFFIX_WORDS — the esq dual membership, deliberate

Fixed with one predicate, `reads_as_suffix`: a suffix piece, or the trailing roman numeral assign's fork takes — and that fork is now `_vocab.is_trailing_numeral_suffix`, lifted out of assign so the two stages share its definition instead of the reserve carrying a copy (the `_ROMAN` pattern moved with it; the regex-sync pin follows it). The reserve counts a piece out when it reads as a suffix, and the join declines the piece it would take when it does — the same decline it already made for a marker. The mirror is assign's fork condition for condition: the numeral is the LAST piece and not the first name piece — not "the piece before it is not a title", which the first draft wrote and the design-docs review falsified, `jr` being title vocabulary as well as a suffix, so `abdul Smith Jr V` lost its family exactly as before — and the piece before it is read as the join would leave it, the joined pair's first token being the bound word (`abdul J. V`, the design-docs review's other find, otherwise suppressed the fork for the reserve alone and read given 'abdul J.', family ''; the reserve now declines, and assign, seeing the unjoined pieces, reads given 'abdul', middle 'J.', family 'V', exactly as it reads `John J. V`). Two more the code review found: assign drops the group-flagged credential pieces (the Ph. D. merge) from its walk at any position before the trailing peel, so a numeral can be last in that walk without being the last piece — `abdul Smith V Ph. D.` lost its family until "last" and "the piece before" were read over the pieces assign keeps — pinned at the piece level only, because a trailing split credential is the shape the 1.4.0 ledger's `[[never]]` entry refuses to explain any diff on, and v1's reading of that name carried the #401 family loss; the example stayed out of the rules corpus rather than the never-entry being narrowed for one bound-word spelling; and the post-comma walk has no numeral fork at all (a trailing numeral there is read by a lenient last-of-two rule, or as a middle initial with a third part), so a mirror applied under LENIENT declined `Berg, abdul V` and `Berg, abdul V, jr` against every baseline with nothing classifying it — the numeral branch is scoped to STRICT, and after a family comma the reserve counts suffix pieces alone, as it always did. So `abdul Smith V Jr` still joins and assign reads family 'V', and `abdul V Smith` still joins an initial, which is a name word. What is NOT mirrored: S2's other positional fork, the bare ambiguous acronym. Where the join leaves two pieces no family is lost — `abdul Smith Ma` reads family 'Ma' where `John Smith Ma` reads suffix 'Ma', assign keeping the acronym as the family of a two-piece name — but with a suffix word between, the join leaves three, assign peels the acronym and then the suffix, and the family is gone: `abdul Smith Jr Ma` reads given 'abdul Smith', family '', suffix 'Jr, Ma' where `John Smith Jr Ma` keeps family 'Smith'. Pre-existing and byte-identical before this fix, and the whole of what remains of #401's shape on the main walk: a review sweep of the STRICT path found 132 family-losing shapes, every one ending in a bare ambiguous acronym. Left for #425 rather than widened into here, and framed there as sharing assign's peel over a post-join VIEW rather than mirroring a second fork by hand: with a view the count is the view's length, the remap and the first-piece carve-out fall away, and the other two sites that stop at a suffix with the same initial veto — P2's chain and M2's walk, which take a trailing numeral the same way (#424) — would ask the question once. The walk was also computed once rather than per piece after review measured the per-piece rebuild as quadratic (13x growth for 4x the input against the benchmark's 6x bound, on a shape no benchmark unit led with); `bound_given` is that unit now. Readings: `abdul Smith V` → given 'abdul', family 'Smith', suffix 'V', the SUFFIX_OR_NAME report unchanged; `abdul Jr Smith Berg` → given 'abdul', middle 'Jr Smith', where `John Jr Smith Berg` puts it; `abdul Ph. D. Smith Berg` → given 'abdul', middle 'Smith', suffix 'Ph. D.' — the SUFFIX field is 1.4.0's again, the given is not: v1's fix_phd extracted the credential before parsing and the join then took 'Smith', reading given 'abdul Smith' — which the issue reports correctly, and then mislabels its own proposed reading as "1.4.0's reading restored". The decline is chosen over reaching past the credential for the next name word, because P5 joins "the word after it": a join that skipped a suffix would read a shape no one writes as if the suffix were not there, and the marker decline already sets the precedent that the join takes the word after it or nothing. A judgement, recorded as one; `Berg, abdul Jr Smith` → suffix 'Jr'; and `Sir abdul V`, the shape #369's licence had widened #401 onto, reads suffix 'V'. #369's licence-local suffix clause is retired, subsumed by the general decline. `bound_given_join_sees_only_the_surviving_name` re-pins to given 'abd', middle 'Jr Smith', as the #418 entry anticipated. No differential corpus name has either shape — measured: the gate was byte-identical at all three baselines before the rules.md examples went in — so the examples carry the shapes into the rules corpus, where they move against every baseline and are classified there. The 1.4.0 `fix(#401)` rule lists three fields where the 2.x rules list two, because 1.4.0 did not even read the V as a suffix; a draft with two fields handed the name to the fields-only suffix-routing catch-all, whose capped printout hid it, and this entry briefly called #401 a 2.0 regression on that inert measurement — the design-docs review caught it. EXPLAINED NOTHING is not byte-identical.

- 2026-08-22 #425 — the reserve runs assign's peel over the view the join would leave. #423 had the reserve re-derive assign's trailing peel by hand — the flagged-credential drop, the walk's start, the numeral fork's conditions, the preceding piece remapped to "as the join would leave it" — with each seam held by a pin, and the bare-ambiguous-acronym fork not mirrored at all: `abdul Smith Jr Ma` read given 'abdul Smith', family '', suffix 'Jr, Ma' where `John Smith Jr Ma` keeps family 'Smith', the whole residue of #401's shape on the main walk (every family-losing bound-word shape in the sweep below contains one). The altitude review's framing is taken: the peel is one function, `_peel_trailing` over the list `_peel_walk` builds (the walk's input defined once too, after the /simplify pass found it spelled at three sites), housed in group beside the piece predicates because assign imports those and group cannot import assign; assign calls it and reports what it returns (the numeral fork's piece, the acronym picks, in the order it always reported them — byte-identical over the suite and the three gates); and the reserve builds the VIEW merge() would leave — the pair one piece, its tags the union — and calls the same function. `BoundJoin`'s "the value IS the threshold" trick survives restated as post-join name pieces: STRICT 2, LENIENT 1. The remap and the `k == first_name_k` carve-out are gone, the joined piece being two tokens and never a suffix piece.

One principle came out of writing the ledger that the view alone did not state. `abdul Smith Ma` had read given 'abdul Smith', family 'Ma' since 2.0, and the #423 entry called that the lossless shape; the 1.4.0 gate's catch-all count said otherwise — 1.4.0 reads first 'abdul', last 'Smith', suffix 'Ma', as `John Smith Ma` reads, and the fields-only catch-all had absorbed the diff since the example went in (the count is the tell, per the #401 entry). Unjoined, the acronym has words to spare and peels as a credential; joined, the view is two pieces and the fork keeps it as the family — the join was CHANGING what assign reads. So the rule is stated as P5 now has it: the join joins two name words into one and changes nothing else, which the code asks directly: the pieces the peel takes are the same either way (a count proxy, "the view leaves exactly one name word fewer", was the first spelling; the altitude review pointed out it held only because the peel is a tail walk, and the comparison states the rule). That comparison subsumes the unjoined-piece check #423 added for `abdul V` (unjoined the peel takes the V, joined nothing) and restores `abdul Smith Ma` to 1.4.0's reading; `abdul Smith Berg Ma` keeps its join, the acronym peeling either way. Every #423 pin holds. Measured: the 1.4.0 gate is byte-identical at the code change, before the examples went in — the three `Ma`-final examples are v1 parity, the catch-all back at 15 — and `abdul Smith Jr Ma` and `abdul Smith Ma` move against 2.0.0 and 2.1.0, classified there; no hand corpus has the shapes. The population the corpus cannot see, from the design-docs review's sweep of 67,860 constructed main-walk shapes (three bound words × twelve words, none a title word, × tails of one to four; no comma shapes): before this change 2,160 lost the family and every one contained a bare ambiguous acronym (864 with further suffixes behind it, so "ending in" was an artefact of #423's shorter sweep); after it none does, and all 67,860 match the family and suffix their ordinary-given twin reads. 5,166 move against master in all — 3,006 where the acronym was the family and is now the suffix (`Sir abdul Smith Ma`), 2,160 where the family is restored. The title-word class below was outside that population and moves after a family comma too (`Berg, abdul Sir`); so was the licence class at the end of this entry.

Three things the principle does not reach, recorded so nobody applies it there. The post-comma LENIENT path is untouched, and NOT for the reason first written here ("its walk has no peel" — it reads a trailing numeral by the lenient last-of-two rule, as the #423 entry says): the family is fixed there and the pair is the given whatever follows, so the principle is not applied, and `Berg, abdul V` keeps given 'abdul V' though the V would be the suffix unjoined — 1.4.0 parity, and exactly the regression #423 measured when a mirror was applied there; P5 now carries it as Accepted. The vocabulary decline is the UNAMBIGUOUS vocabulary: a bare ambiguous acronym the peel does not take is a name word, so `abdul Ma Smith` reads given 'abdul Ma' (1.4.0 read middle 'Ma' — a 2.x difference no corpus witnesses, now an Accepted example). And the principle is about the suffix reading, not the report: `Sheik abdul Ma` reports no SUFFIX_OR_NAME where `Sheik John Ma` does, the #369 precedent — a bound word read as the bound word is not a fork. One more class moves, found by the code review and then measured against 1.4.0: the 2.x reserve had excluded TITLE pieces from its count — an unrecorded deviation from v1, whose join counted suffixes out and nothing else — and the shared peel, following assign, reads a mid-name title word as the name word it is. `Berg, abdul Sir` (given 'abdul', middle 'Sir' since 2.0) and `abdul Smith Sir` (declined since 2.0) read given 'abdul Sir' and given 'abdul Smith', family 'Sir' — 1.4.0's readings, both; `abdul Sir Smith Berg` never stopped reading given 'abdul Sir'. No corpus name has the shapes; P5 says it now and `abdul Sir Smith Berg` is an example. What the review actually caught was the tag artefact that exclusion had been hiding: the conjunction merge derives a `title` piece tag for a chain such as "Sheikh and Ahmad", merge()'s union handed it to the pair, and assign peeled the pair as a leading title — `abdul Sheikh and Ahmad Bakar Smith` read title 'abdul Sheikh and Ahmad' on 2.0 and 2.1 (v1 read it as the given), and `abdul Sheikh and Ahmad Bakar`, which the exclusion had kept unjoined, would have too once it went. The bound join drops the tag, and the view drops it the same way — the pair is a given name whatever tag the word carried. One more class the comparison moves, found by the code review's classification of every residual master-to-branch diff (12 of 17,640 constructed names, one shape; 360 of 360 in a dedicated sweep): behind a given-name title, a numeral after an initial-shaped piece. `Sir abdul J. V` read given 'abdul J.', suffix 'V' since #369 — the fork suppressed unjoined, fired joined, and the licence's threshold of one let the join through — and the comparison declines it: given 'abdul', middle 'J.', family 'V', exactly as `Sir John J. V`. Pinned at both levels; it was the only shape a looser comparison let through under mutation at the test review. #424 — P2's chain and M2's walk taking a trailing numeral — is the remaining asker of the same question and now has `_peel_trailing` to ask it of.

Excluded (BOUND_GIVEN_NAMES):

- mohamad — a standalone given name in its own right; binding it would eat the middle name.
Expand Down
40 changes: 29 additions & 11 deletions docs/design/rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -254,15 +254,23 @@ P5. Rationale: some given-name words are incomplete alone — "abdul"
not among the words to spare: they leave the name, so counting
them asks the question about a name that will not exist. The join
never absorbs a marker standing as a word of its own — a marker
is not a name word (M2) — nor a word of the suffix vocabulary
(S2), wherever position will then place it; a marker left as a
word that a particle join (P2) has already taken travels with
that join. The reserve counts the same words out, and one more
that position decides: a trailing roman numeral that assign
reads as the suffix (S2) is no word to spare, and is not joined
— read as assign will read it, after the join, with the bound
word first. After a family comma there is no such fork to read:
the reserve there counts suffix words alone.
is not a name word (M2) — nor a word of the unambiguous suffix
vocabulary (S2), wherever position will then place it; a bare
ambiguous acronym is a name word wherever the peel does not take
it; a marker left as a word that a particle join (P2) has already
taken travels with that join. A title word standing in the name
is a name word (H3) and joins like one: the pair is a given name
whatever tag the word carried. What there is to spare is what
assign will leave: the join is tried on the pieces as it would
leave them, assign's trailing peel (S2) is read over that, and
the name words it leaves are the words to spare — a trailing
roman numeral, or a bare acronym with words behind it, is no
word to spare. The join joins two name words into one and
changes no suffix reading: a word the peel reads as a suffix
unjoined must read so joined, or the join declines. After a
family comma the family is fixed and the joined pair is the given
whatever follows, so the reserve there reads no peel: the join
stands whenever the word after the bound word is a name word.
"abdul salam ahmed salem" → given="abdul salam"
"abd Allah Smith" → given="abd Allah"
"Salam, abd Allah" → given="abd Allah"
Expand All @@ -280,7 +288,11 @@ P5. Rationale: some given-name words are incomplete alone — "abdul"
"abdul Smith V" → family="Smith"
"abdul Smith V" → suffix="V"
"abdul Smith Jr V" → family="Smith"
"Berg, abdul V" → given="abdul V" · boundary
"abdul Smith Jr Ma" → family="Smith"
"abdul Smith Jr Ma" → suffix="Jr, Ma"
"abdul Smith Ma" → suffix="Ma"
"abdul Smith Berg Ma" → family="Berg" · boundary
"abdul Sir Smith Berg" → given="abdul Sir"
"abdul Jr Smith Berg" → given="abdul"
"abdul Jr Smith Berg" → middle="Jr Smith"
"abdul Ph. D. Smith Berg" → suffix="Ph. D."
Expand All @@ -289,6 +301,12 @@ P5. Rationale: some given-name words are incomplete alone — "abdul"
"abd Allah Smith née Jones" → given="abd Allah"
"abd née Jones" → given="abd"
"Berg, abd née Jones" → suffix="abd"
Accepted: after a family comma the join stands though, unjoined,
assign would read the word it takes as the suffix — the family
is fixed there and the pair is the given; and a bare ambiguous
acronym the peel does not take joins as any name word does.
"Berg, abdul V" → given="abdul V"
"abdul Ma Smith" → given="abdul Ma"
Accepted: a given-name title plus a bound pair leaves the family
empty, as H1 accepts for "Sir John" — the input names no family.
Under a family-first order the joined pair is the family and the
Expand Down Expand Up @@ -401,7 +419,7 @@ S2. Rationale: generational suffixes and credentials are recognized
"Jack Wei Ma" → suffix="Ma"
"Jack Wei Ma" → ambiguities=("suffix-or-name",)
"Smith Jr." → family=""
implemented: nameparser/_pipeline/_classify.py, nameparser/_pipeline/_vocab.py
implemented: nameparser/_pipeline/_classify.py, nameparser/_pipeline/_group.py, nameparser/_pipeline/_vocab.py

S3. Rationale: credentials are often written run together with
periods; the chunks between the periods are what carry the
Expand Down
Loading