Keep a never-given particle attached under any name_order (#359) - #361
Merged
Conversation
post_rules rule 1b folds a name that opens with a particle that is never a given name into the family: "de Mesnil" is all surname. It asked for that particle by ROLE -- the single GIVEN token -- and under Policy(name_order=FAMILY_FIRST) the opening particle is already FAMILY and the GIVEN role belongs to the token behind it. So the tag test inspected the wrong word, found no particle, and declined: family 'de', given 'Mesnil'. The rule's intent is order-agnostic and its implementation was not. The decision recorded in #359 is that a never-given particle keeps its particle whatever order the caller declared: a word that can never be a given name leaves name_order nothing to place, so declaring family-first is not a reason to make 'de' a surname on its own. Only the trigger moves; the fold itself (givens and middles into family) was already written in roles rather than positions and reads correctly under any order. _leading_name_piece reads the opening name piece off `pieces` instead: the first piece carrying a name role, skipping the titles and group-flagged suffixes assign already peeled, in segment 0 -- or in segment 1 under a family comma, where segment 0 is fixed as the surname and the name continues after the comma. That last clause is what keeps the family-comma reading identical rather than gating it off: master folds 'Smith, de Mesnil' into family 'Smith de Mesnil' TODAY, in the default order, so a NO_COMMA-only gate would have been a default-order change, not a guard against one. The two conditions the old trigger carried survive in position form. A single-token piece is what `len(givens) == 1` was saying: a particle group already chained forward ('Mr. de Mesnil' is one piece) is not a lone leading particle, and a title makes the particle non-leading, so the chain has it. And there must be another name token to fold, which leaves a degenerate bare 'de' as it stands rather than inventing a surname. Every default-order parse is unchanged, which is the property the change turns on: the differential harness reports byte-identical output over all 751 corpus names against 1.4.0, 2.0.0 and 2.1.0 alike. One shape moves the other way, and its test says so: 'Mesnil de' under FAMILY_FIRST put a bare 'de' in the given POSITION, which the old role test folded. The rule is about a leading particle and 'Mesnil' is not one, so the position-keyed form declines it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four sites asserted, correctly until the previous commit, that Policy(name_order=FAMILY_FIRST) reads "de Mesnil" as family 'de', given 'Mesnil'. #354 wrote two of them and #358 scoped the other two to the default order precisely so they would not bless that output. Now that the fold is order-independent, the first two are false and the scoping on the others is over-cautious in the one place a reader is most likely to test it. - config/particles.py: both docstrings. NON_GIVEN_NAME_PARTICLES no longer scopes its "the whole thing is a surname" reading to the default order, and PARTICLES now splits the sentence where the behavior splits -- a leading member is the surname under every order, while a leading particle OUTSIDE the set is genuinely order-dependent and still reads as family 'van', given 'Gogh' under FAMILY_FIRST. - _lexicon.py: particles_ambiguous said a non-member is folded "under the default given-first order". It is folded under all of them; what stays name_order's question is where a MEMBER's piece lands. - AGENTS.md's config-layer entry, same correction, plus the mechanism (the fold keys on position) since that is what a future reader needs to know before touching the rule. - docs/customize.rst and docs/usage.rst: no false claim to fix, but both fenced the reading off to the default order in a sentence that is now about two different things. Each says which half name_order still governs. customize.rst's particles_ambiguous section gets the point of #359 as well: taking a word out of the set now changes the parsed fields under a family-first order, where before it moved only the ambiguity report. Every replacement claim was measured by parsing, not reasoned: FAMILY_FIRST gives family 'de Mesnil' and 'de la Vega' with no given name, family 'van' + given 'Gogh' for "van Gogh", family 'de' for the bare "de", and family 'van Gogh' once 'van' is removed from particles_ambiguous. FAMILY_FIRST_GIVEN_LAST agrees with FAMILY_FIRST on all of them. The release log bullet records the FAMILY_FIRST change and the "Mesnil de" shape that moves the other way; the 2.2.0 preamble's "no parse changes" is narrowed to the default order, which is what its own 751-name measurement covers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… rule (#359) The re-key in dacc700 traded one shape of a single invariant for another. Keying on the opening position fixed "de Mesnil" under FAMILY_FIRST and lost "Mesnil de", where a bare 'de' sits in the given POSITION -- the trailing piece under that order -- and used to fold by the role test. The branch reported given='de' for it, which contradicts the vocabulary the rule consults: NON_GIVEN_NAME_PARTICLES exists to say that word is never a standalone given name. The two shapes are not two rules. The invariant is: a never-given particle is never REPORTED as the given name and the repair is identical in both -- given and middles join the family. Opening the name, the particle pulls the rest of it in ("de la Vega"); left alone in the given position, it folds into the family beside it ("Mesnil de"). That is why the old role test caught the trailing case seemingly by accident: under the default order the opening piece IS the given, so one test covered both. Only under a family-first order do the two come apart, and both need asking. So the rule now asks at both sites -- the opening piece from `pieces`, and the given-position tokens -- with the single-token and another-name-token-to-fold-with guards shared. The comment and the stage header state the invariant first and the two shapes under it, rather than presenting the second site as a legacy arm. 'Mesnil de' is pinned under BOTH orders, with a comment saying what it protects: it is the shape a refactor reading the rule as leading-particle-only drops, silently and under a non-default order. Verified rather than asserted, since "an extra site can only add firings" is exactly the kind of claim that is true until it isn't: the differential harness reports byte-identical output to master over all 751 corpus names at 1.4.0, 2.0.0 and 2.1.0. Against the branch's own pre-restore measurements, one line of the 34-name probe matrix moves -- 'Mesnil de' under FAMILY_FIRST, from family='Mesnil' given='de' back to family='Mesnil de' -- and nothing else, in either order. Prose that had inherited the leading-only framing moves with it: config/particles.py's NON_GIVEN_NAME_PARTICLES docstring, AGENTS.md's config-layer entry (which now records the misreading itself, since it is how the bug got in), and the release-log bullet, which no longer announces the 'Mesnil de' regression it no longer has. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #361 +/- ##
=======================================
Coverage 98.50% 98.51%
=======================================
Files 44 44
Lines 2883 2894 +11
=======================================
+ Hits 2840 2851 +11
Misses 43 43 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Four documents claim rule 1b for every `name_order` and one order was never parsed. `FAMILY_FIRST_GIVEN_LAST` folds a leading never-given particle exactly as `FAMILY_FIRST` does -- measured over the whole differential corpus, the same seven names move under each -- so the family-first cases are parametrized over both orders rather than duplicated. `de Mesnil Garcia` joins them: it is one of those seven, and the only no-comma family-first shape whose fold has a MIDDLE to move as well as a given. `_leading_name_piece`'s skip loop had no test. Two mutations survived the entire suite -- dropping its name-role filter, and reading only `pieces[seg][:1]` -- because the one case with a title in front, `Mr. de Mesnil`, chains the particle into a two-token piece where the rule declines with or without the skip. A mid-name suffix word breaks that chain: `Dr. de MD Mesnil` leaves the particle a lone piece behind the title piece, and both mutations then split it into given `MD`, middle `Mesnil`, family `de`. Both now fail. Every other case rides on the fixture lexicon's `de`. The rule is claimed of the whole never-given class, so sweep it off the live `Lexicon.default()` in all three orders, the way `test_properties.py` already sweeps `particles_ambiguous` -- asserting the behaviour, not the membership, so an addition to the set is covered the day it lands. Rule 1 retags GIVEN->FAMILY without recomputing the role index lists, so `givens`/`middles`/`families` are stale by the time rule 1b reads them: over the 751 corpus names in four policies, that arm fires 48 times and the lists are stale at 1b on all 48. Harmless today -- 1b fires on none of them -- but it is the shape of the bug this branch fixes, a guard reading a token list that no longer means what its name says. Recompute, as 1b already does after its own fold. Parse output is byte-identical afterwards in all three orders over the corpus. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Five documents stated the rule as "a particle that is never a given
name is never *reported* as the given name", and this branch's own
test file ships four counterexamples: `parse("de").given` is `'de'`,
`parse("Sir de Mesnil").given` is `'de Mesnil'` in the default order,
and under FAMILY_FIRST `"Juan de la Vega"` reports given `de la Vega`
-- which #359 blesses in as many words. What the code enforces, and
what the `len(site) == 1` guard says, is one clause narrower: where a
never-given particle stands ALONE as a piece, opening the name or in
the given position, the name is left with no given name at all, the
given and the middles folding into the family -- as long as there is
another name token to fold into. Checked over 3,591 parses (751
corpus names plus 446 synthetic shapes, three orders): the guard holds
in 303 of them and the parse has no given and no middle in all 303,
against 82 failures of the same check on the pre-fix tree.
The release note named one non-default order. Both change, and by the
same seven of the 751 corpus names -- a reader on
FAMILY_FIRST_GIVEN_LAST would have concluded they were unaffected.
Smaller corrections, each re-measured by parsing:
- "ambiguous 'van Gogh' keeps its given reading" is default-order
only, and sits in the paragraph explaining that the rule stopped
being order-scoped. Under either family-first order `van` is the
family.
- "a name *starting* with one of these has no given name, under EVERY
name_order" over-reaches for the same reason `Sir de Mesnil` does.
It is the opening PIECE that is asked about.
- "'Mr. de Mesnil' is one piece" is false -- it is two, `((0,),
(1, 2))`. The intended claim is about the particle group; 1b
declines on both sites there, and the family reading is rule 1's in
the default order.
- the stage header still said `Consumes: tokens (roles assigned)`
after this branch made post_rules a consumer of `pieces`, and it
read `structure` unannounced before that.
- `_leading_name_piece` skips by ROLE, so NICKNAME, MAIDEN and unroled
pieces are skipped too, not just "titles and group-flagged
suffixes"; and it has two empty exits, not one.
- the family-comma test credited `assign` for a fold rule 1b does.
- "EITHER order" at three sites: there are three orders.
`docs/customize.rst` claimed positional input is assigned in the order
you declare, which now has a second exception; it joins the
`Nguyen Van Minh` caution already in that section.
Not done, deliberately: the "under every name_order" prose in
customize.rst and usage.rst sits beside bare `parse(...)` doctests,
and a family-first line in each would make the claim executable under
`sphinx -b doctest`. The repo's convention is prose plus a unit test
rather than a doctest block, and the previous commit's sweep already
executes exactly that claim over the whole never-given class in all
three orders -- so the doctests stay as the vocabulary examples they
are.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ns (#359) `docs/customize.rst` and `docs/usage.rst` carried the same first-word-versus-lone-piece over-reach the previous commit corrected in `config/particles.py` and `AGENTS.md` -- "one starting with an unlisted particle has no given name", "position matters in exactly one place: the start of a name". Both are falsified in the DEFAULT order, so neither needed a family-first parser to catch: `Sir de Mesnil` reports given `de Mesnil`, because the title pushed `de` off the front and it chained onto `Mesnil` there, and `de Mesnil, Juan` reports given `Juan`, because the comma named the surname first. Each page now says standing ALONE at the front, in its own register, and both counter-examples go into the adjacent doctest block rather than into prose -- they are bare `parse(...)` calls in the default order, which is exactly what those blocks already are, so `sphinx -b doctest` now executes the qualification instead of the reader taking it on trust (223 -> 227 doctests). The rule-1b comment now records what the rule does NOT cover, since that is where a reader will ask. MIDDLE is deliberately outside its two sites and shows it -- `Mesnil Garcia de` strands `middle='de'` under FAMILY_FIRST while FAMILY_FIRST_GIVEN_LAST, whose given position is that same trailing piece, folds the whole name to `family='Mesnil Garcia de'` (#365). And how much the fold takes once it fires is the other open question: `de Mesnil Juan` goes wholly to the family in every order, matching the default rather than stopping at the particle group (#364). No logic changed; parse output over the 751 corpus names is identical in all three orders. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The previous commit scoped two doc sites with a pair of
counter-examples, and pinned both as doctests. One of them is now
believed to be a bug: `parse("Sir de Mesnil")` reports given
`de Mesnil` with family `''`, and a particle chain is a surname by
construction -- `Lexicon.particles` chains onto the following piece to
BUILD a family name, and `NON_GIVEN_NAME_PARTICLES` exists to say its
members can never be a given name. A chain sitting in `given`, with no
surname at all, contradicts that vocabulary rather than qualifying it.
It is filed as #367. The mechanism is a collision, not a decision
about particles: `sir` is in `GIVEN_NAME_TITLES`, which suppresses
post_rules rule 1, so the chain is left in the given position --
`Dr.`, `Lady` and `Prof.` all give family `de Mesnil`. Longstanding
rather than a 2.x regression: `nameparser==1.4.0` from PyPI gives
title `Sir`, first `de Mesnil`, last `''`.
So the previous commit presented a bug as a deliberate design
qualification, and made doctests of it -- which would both read wrong
and FAIL the moment #367 lands. Removed here rather than left to be
found then. What replaced it in each page is the comma alone, which is
correct under any reading and which #367 does not touch: a comma names
the surname before a leading particle decides anything, so what
follows it is the given name. Each page keeps its own register, and
the surrounding "standing alone" framing stays -- it is a sufficient
condition, still true after #367, not the exclusivity claim that was
the over-reach.
`config/particles.py`, `AGENTS.md` and the rule-1b comment cited the
case from 2309783 as JUSTIFICATION for the `len(site) == 1` guard's
shape. It is still worth naming there, because mechanically it IS
what that guard does -- so it is named as suspected-wrong with a
pointer to #367, and each site now says outright not to cite it as a
line the rule means to draw. The reasoning itself now rests on the two
sound counter-examples: `Juan de la Vega`, whose given position under
FAMILY_FIRST holds a three-token chain rather than a lone particle,
and the degenerate bare `de` with nothing to fold into. The first is
stated as what the guard tests rather than as a blessed output, since
#367 disputes that row too even though #359 records it as intended.
Comments and doctests only; no executable line changed. Parse output
is identical -- verified over `Sir de Mesnil`, `Dr. de Mesnil`,
`Lady de Mesnil`, `Prof. de Mesnil`, `de Mesnil, Juan`, `de Mesnil`,
`Juan de la Vega` and bare `de` in all three name orders. Doctests
227 -> 225, exactly the two removed `Sir de Mesnil` assertions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #359.
A never-given particle stopped attaching to the surname under
Policy(name_order=FAMILY_FIRST), so a documented configuration option silently did nothing and a name split at the particle:Uniform across the whole class: all 28
NON_GIVEN_NAME_PARTICLESmembers split underFAMILY_FIRSTand none under the default. The 39 may-be-given particles are untouched —van Goghstill gives familyvan, givenGoghunder family-first, and the vocabulary line between the two sets is #360's question, not this PR's.The fix
post_rulesrule 1b keyed on theGIVENrole. Under the default the opening piece is the given, so one role test caught everything; underFAMILY_FIRSTthe opening piece is the family and the given sits behind it, so the test inspected the wrong token and declined.The rule now states the invariant it actually enforces — a particle that is never a given name is never reported as the given name — and asks for it at the two sites that invariant can be violated: the piece that opens the name (read from
pieces, so order-independent) and a particle left alone in the given position. Both guards are shared, so it reads as one rule with two shapes rather than two rules.That framing matters. A first pass keyed only on the opening piece, which is the literal reading of the issue, and it lost a case the old role test caught by accident:
"Mesnil de"underFAMILY_FIRSTbecamegiven='de'— from the set whose entire meaning is "never a given name". Now pinned by a test that says what it protects.Verification
The load-bearing property is that default-order parsing is untouched. Measured, not assumed —
tools/differential/compare.pyat three baselines, with the full reports diffed against agit archiveof master, not just the counts:Shapes that were already correct and stay so:
Dr. de Mesnil(title at piece 0, particle chains at piece 1),de Mesnil, Juan(family comma already fixes the surname),Smith, de Mesnil(folds under a family comma in both orders on master — a structure gate would have broken that), and barede(nothing to fold into).Review round
A four-agent review (comments, tests, general code, silent failures) found the rule itself correct, and established it more strongly than the PR originally claimed: the guard rewrite was proven equivalent to the old one for the pre-existing arm by instrumenting 374,031 parses (0 mismatches), and the position site was shown to be a strict subset of the givens site in the default order — the divergence row has count 0. Default-order output is unchanged across three independent corpora: 751 real names, 18,048 synthetic, 124,677 constructed.
Everything it found was prose or coverage. Three follow-up commits:
_NAME_ROLESfilter in_leading_name_pieceand its piece scan — on a shape reachable in 48 of 28,593 synthetic names (Dr. de MD Mesnil, where a mid-name suffix word breaks the chain that would otherwise swallow the particle). Now killed by a test and nothing else.FAMILY_FIRST_GIVEN_LASTwas claimed in four documents and parsed in none; the family-first tests are parametrized over both orders. And the never-given class was exercised through one word, whichAGENTS.mdnames as the anti-pattern — replaced with a sweep over the liveLexicon.default()set in all three orders.parse("de").givenis'de',parse("Sir de Mesnil").givenis'de Mesnil', andJuan de la Vegaunder family-first givesgiven='de la Vega', a case issue particles_ambiguous has no effect on the parse under name_order=FAMILY_FIRST #359 explicitly blesses. The claim now matches thelen(site) == 1guard: where a never-given particle stands alone as a piece, the name is left with no given name at all, provided there is another name token to fold into. Verified over 3,591 parses — 303 qualifying cases, 0 violations, against 82 violations on the pre-fix tree.docs/customize.rstanddocs/usage.rstcarried the same first-word-vs-lone-piece over-reach, falsified in the default order byparse("de Mesnil, Juan").given == 'Juan'— a name that starts withdeand has a given name. Both now scoped, with the counter-examples as doctests rather than prose, so CI executes them (223 → 227 doctests).The release note said one non-default order changes; both do, identically, and the same seven corpus names move under each. Fixed.
Two questions this deliberately does not settle, both cited in the rule's own comment: #365 (a never-given particle landing in MIDDLE is stranded under
FAMILY_FIRSTand folds underFAMILY_FIRST_GIVEN_LAST) and #364 (how much the fold takes once it fires).Documentation
config/particles.py,AGENTS.mdand_lexicon.pyasserted thatFAMILY_FIRSTsplitsde la Vegainto familyde/ givenla Vega. That was true when written, in #354 and #358, and this PR falsifies it — #359 listed both as docs that must move with the behaviour. Each was re-verified by parsing rather than reasoning.AGENTS.mdalso records the misreading itself, since reading the rule as leading-only is how the bug got in.The 2.2.0 preamble's "no parse changes" is narrowed to "no parse changes in the default name order", which is all its 751-name measurement ever covered.
🤖 Generated with Claude Code