Skip to content

Commit 11a486d

Browse files
derek73claude
andcommitted
refactor(group,docs): the cleanup round, and the pin the guard was missing
/simplify, four angles. Reuse and efficiency returned no changes; the other two found real work. The one that matters: the diff's central conjunct was UNPINNED. Dropping `in_entry and` from the tag test left the whole suite green -- including the guard added for exactly this -- while diverging on 1,152 of 65,725 inputs. That guard pins the sticky entry_open update, not the tag condition. The distinguishing input needs a suffix piece FIRST and then TWO titles, so the second title continues an entry it does not belong to: "Smith, MD Rev. Dr." gives title_list ['Rev.','Dr.'] and the mutant gives ['Rev. Dr.'], with the title STRING identical either way. Added, and verified to fail against the mutation copy. The invariant this change rests on -- one comma segment is one suffix entry -- was stated in NO rule. C1 governs routing; R1 governed word ORDER and never mentioned separation; the only statement was a code comment. That is a large part of why two stages disagreed about it for years, so R1 gains the sentence and two examples, both names already in the corpus (a third, "Smith, MD, PhD", would have moved for a #325 routing reason and been misfiled under #429). Also from the review: - Two case rows removed. family_comma_three_credential_entries is a strict kill-subset of the title-led row AND its note claimed to pin a latch that family_comma_title_between_credentials actually pins -- all three of its pieces are in the entry, so the latch never has to survive a gap. family_comma_title_run_does_not_join kills nothing, by its own note; test_facade.py holds the same input with assertions that can see the collapse. - `family_comma` is redundant by invariant (segment() emits at most one segment for NO_COMMA; SUFFIX_COMMA short-circuits on `tail`). Kept for locality, now documented as unpinnable rather than left to look like missing coverage. - I reintroduced #191 in my own new comment while this same PR corrects it elsewhere. Fixed here and at the two remaining delimiter sites. test_group.py:128 keeps #191 -- it is genuinely about the prefixes. - The predicate moved below _is_suffix_piece, which it calls, restoring the _is_leading_title/_leading_titles pair it had split. - The "whatever role it holds" claim is qualified: no multi-token TITLE piece witnesses that branch in 38,892 generated inputs. - The docstring's regression narration trimmed to the contract sentence plus a pointer; the concrete account lives at the call site. Efficiency recorded rather than optimised: _segment_holds_no_name runs twice per family-comma parse, measured pure recomputation (408 of 506 comma-bearing literals reach both sites, 0 disagreements). Worth 1.2-2.2% of a family-comma parse and 0% of every other. Caching needs a derived ParseState field the sites would not otherwise share, so decisions.md carries the number. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent beeb7c0 commit 11a486d

6 files changed

Lines changed: 74 additions & 62 deletions

File tree

docs/design/decisions.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -442,6 +442,8 @@ Excluded (MAIDEN_MARKERS, per nameparser/config/maiden_markers.py):
442442
SCOPE, stated as the condition actually reads rather than as the first draft described it: any family comma whose following segment holds no name word, NOT only a one-word family. `John Smith, Jr. III` renders suffix 'Jr. III' (1.4.0's reading), and a title-led segment is in too — `Smith, Dr. MD PhD` renders 'MD PhD'. The differential cannot witness the widening: of 1069 corpus names, 303 carry a structure comma, the predicate is true for 36, only 9 of those have two or more pieces in the segment, and none of the 9 combines a multi-word pre-comma name with two suffix pieces. 0 unexplained is therefore evidence about the corpus, not about the class — the case table carries it instead.
443443
THE REGRESSION THE REVIEW FOUND, and the reason this entry is long: the first draft let ANY piece open an entry, as the tail block always had. That is safe on a tail segment, where assign routes every piece to SUFFIX — which is what `tail` means — and wrong off it, where a title piece routes to TITLE. Two failures, one cause. The `joined` tag is role-BLIND and the facade heals it for every role, so a title opening an entry collapsed `title_list` from ['Rev.', 'Dr.'] to ['Rev. Dr.'] on `Smith, Rev. Dr.`; and where the pre-comma name left a suffix, the tag glued the next suffix BACKWARD across the writer's own comma — `Smith Jr., Mr. Jr.` rendered suffix 'Jr. Jr.', the exact inverse of the bug being fixed. Neither is visible to the differential, which compares strings, nor to the case table as it stood, which asserts the title STRING (space-joined either way). The rule that replaces it separates two joins that had been one: WITHIN a piece the tag renders a merged piece as one unit, whatever role it holds; BETWEEN pieces it continues an entry, and only a piece that renders into the same run may do that. The entry is sticky across a piece that is not in it, so an interleaved title does not split the run it sits in (`Smith, MD Dr. PhD` renders 'MD PhD'); a delimiter core still closes it, which is the one thing that separates two entries.
444444
Scoped deliberately, and the scoping is the other decision: `tail` still gates delimiter-core dropping, which is v1 `expand_suffix_delimiter` parity (#206, commit 021823e — three code comments say #191, which is the German/Dutch vocabulary PR, and are corrected here) and a TAIL rule. The core set is threaded to the block as `seg_cores` so the widened condition cannot drop cores where `tail` says not to. Worth knowing before anyone tries to pin that: for every realistic delimiter the two spellings are indistinguishable by construction, because the core must first pass the very predicate that gates the block — the only reachable class is a policy whose delimiter core is itself suffix or title vocabulary (`Policy(extra_suffix_delimiters={" jr "})`, `Smith, MD jr`), which the case table does not enshrine.
445+
Cost, measured rather than optimised away: `_segment_holds_no_name` now runs TWICE per family-comma parse, once here and once in assign, on the same pieces — verified pure recomputation, not two questions (408 of 506 comma-bearing test literals reach both sites, 0 disagreements, and off-tail `seg_cores` is empty so `kept` never filters, which is why assign sees exactly the pieces group tested). Removing the second call is worth 0.6–1.4 µs, 1.2–2.2% of a family-comma parse and 0% of every other parse (NO_COMMA and SUFFIX_COMMA make no call at all). Caching it needs a derived ParseState field the two sites would otherwise not share, so the number is recorded instead. The order is cheapest-first per 7009fcf: two bool loads and an int compare gate the call.
446+
R1 gained the separator sentence with this change. That the invariant it states — one comma segment is one suffix entry — was written down in no rule is a large part of why two stages disagreed about it for years: C1 governs routing, R1 governed word ORDER and never mentioned separation, and the only statement of the rule was a code comment.
445447
Left alone, measured and pre-existing: the NO-COMMA path comma-joins a space-separated run too (`John Smith MD PhD` renders 'MD, PhD' at 1.4.0, 2.0.0, 2.1.0 and master alike). The fix requires FAMILY_COMMA and does not reach it; that path has no writer-drawn boundary and is a separate question, recorded here and NOT yet filed. It is load-bearing for one thing: `str()` of a fixed parse is a no-comma string, so re-parsing it re-inserts the comma — round-tripping is not stable on these shapes, and an earlier draft of the release bullet claimed it was.
446448
The name `Smith, MD PhD` entered the corpora as this fix's C1 example — a FIRING one, carrying no `· boundary` annotation, which marks the non-firing example a rule must have — so its ledger entries arrive with it. A fourth written for the 1.4.0 ledger EXPLAINED NOTHING and was dropped: `fix(comma-family) lone post-comma piece routes to suffix/title, not first` already claims the name there, its fields covering all three that move. That is #373's dormant-rule diagnosis doing its job, and #426 is the precedent for dropping rather than keeping a shadowed rule.
447449

docs/design/rules.md

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -875,10 +875,17 @@ R1. Rationale: a field is a way of reading the parse, not a stored
875875
words — O3's fold and, since #379, P6's attached tussenvoegsel —
876876
which render before the rest of the family wherever they stood in
877877
the string.
878+
Words are separated as the writer separated them. The suffix view
879+
is the one place this is visible, because it is the only field
880+
that can hold parts the writer comma-separated: a run of
881+
post-nominals written with spaces renders with spaces, and one
882+
written with commas keeps them.
878883
"Dr. Juan Q. Xavier de la Vega III" → family="de la Vega"
879884
"Hassan, Mohamad Ahmad Ali" middle_as_family → family="Ahmad Ali Hassan"
880885
"Hassan, Mohamad Ahmad Ali" → family="Hassan" · boundary
881-
implemented: nameparser/_types.py
886+
"Smith, MD PhD" → suffix="MD PhD"
887+
"John Smith, MD, Bart" → suffix="MD, Bart"
888+
history: decisions.md#C1 · implemented: nameparser/_types.py
882889

883890
R2. Rationale: callers need the surname with and without its
884891
particles — sorting wants "Vega", display wants "de la Vega".

nameparser/_pipeline/_group.py

Lines changed: 51 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -102,44 +102,6 @@ def _is_leading_title(piece: Sequence[int], ptags: Set[str],
102102
and bool(_PERIOD_ABBREV.match(tokens[piece[0]].text)))
103103

104104

105-
def _segment_holds_no_name(pieces: Sequence[Sequence[int]],
106-
ptags: Sequence[Set[str]],
107-
tokens: Sequence[WorkToken]) -> bool:
108-
"""The segment is titles and suffixes only ('John Smith, Dr.',
109-
'John Smith, Mr. Jr.') -- nothing in it is a name word.
110-
111-
The FAMILY_COMMA rule "segment 0 is wholly the family name" rests on
112-
the writer having said where the family name ends. A comma followed
113-
by no name word said no such thing -- 'John Smith, Dr.' is 'Dr. John
114-
Smith' with the honorific moved, and 'John Smith, Mr. Jr.' the same
115-
with the postnominal along -- so the pre-comma name keeps its
116-
positional read instead of being merged. Uses the same
117-
_is_leading_title predicate the peel does, period-abbreviation
118-
inference included, so the two cannot disagree about what a title
119-
is; a suffix piece counts as what it is, so a mixed run like
120-
'Smith, Dr. Jr.' is a title and a postnominal, each read where it
121-
stands, and never a title run 'Dr. Jr.'. An empty segment
122-
('Doe,, Jr.') holds no title to read by.
123-
124-
TWO callers, asking it for different reasons, and the difference
125-
matters. assign uses it to decide whether the comma fixed the family
126-
name (above). group's one-entry join (#429) uses it to decide
127-
whether the segment is a credential run at all.
128-
129-
True does NOT mean "every piece is a suffix" -- the title tolerance
130-
is the whole point, and a true segment can still hold pieces assign
131-
routes to TITLE. A caller that renders the segment as one unit must
132-
therefore ask _is_suffix_piece per piece as well; assuming otherwise
133-
is what made #429's first draft collapse title_list and glue a
134-
suffix across a comma the writer typed.
135-
"""
136-
if not pieces:
137-
return False
138-
return all(_is_suffix_piece(pieces[k], ptags[k], tokens)
139-
or _is_leading_title(pieces[k], ptags[k], tokens)
140-
for k in range(len(pieces)))
141-
142-
143105
def _leading_titles(pieces: Sequence[Sequence[int]],
144106
ptags: Sequence[Set[str]],
145107
tokens: Sequence[WorkToken]) -> int:
@@ -183,6 +145,44 @@ def _is_suffix_piece(piece: Sequence[int], ptags: Set[str],
183145
return "vocab:suffix" in tags and "initial" not in tags
184146

185147

148+
def _segment_holds_no_name(pieces: Sequence[Sequence[int]],
149+
ptags: Sequence[Set[str]],
150+
tokens: Sequence[WorkToken]) -> bool:
151+
"""The segment is titles and suffixes only ('John Smith, Dr.',
152+
'John Smith, Mr. Jr.') -- nothing in it is a name word.
153+
154+
The FAMILY_COMMA rule "segment 0 is wholly the family name" rests on
155+
the writer having said where the family name ends. A comma followed
156+
by no name word said no such thing -- 'John Smith, Dr.' is 'Dr. John
157+
Smith' with the honorific moved, and 'John Smith, Mr. Jr.' the same
158+
with the postnominal along -- so the pre-comma name keeps its
159+
positional read instead of being merged. Uses the same
160+
_is_leading_title predicate the peel does, period-abbreviation
161+
inference included, so the two cannot disagree about what a title
162+
is; a suffix piece counts as what it is, so a mixed run like
163+
'Smith, Dr. Jr.' is a title and a postnominal, each read where it
164+
stands, and never a title run 'Dr. Jr.'. An empty segment
165+
('Doe,, Jr.') holds no title to read by.
166+
167+
TWO callers, asking it for different reasons, and the difference
168+
matters. assign uses it to decide whether the comma fixed the family
169+
name (above). group's one-entry join (#429) uses it to decide
170+
whether the segment is a credential run at all.
171+
172+
True does NOT mean "every piece is a suffix" -- the title tolerance
173+
is the whole point, and a true segment can still hold pieces assign
174+
routes to TITLE, so a caller rendering the segment as one unit must
175+
ask _is_suffix_piece per piece as well. What assuming otherwise cost
176+
is recorded at the one-entry join in group(), the caller that made
177+
the assumption.
178+
"""
179+
if not pieces:
180+
return False
181+
return all(_is_suffix_piece(pieces[k], ptags[k], tokens)
182+
or _is_leading_title(pieces[k], ptags[k], tokens)
183+
for k in range(len(pieces)))
184+
185+
186186
class Peel(NamedTuple):
187187
"""What assign's trailing peel made of a walk. `names` is a count
188188
of positions in the caller's `rest`: rest[:names] are the name
@@ -905,6 +905,13 @@ def group(state: ParseState) -> ParseState:
905905
# never typed, where the full-name 'John Smith, MD PhD' has
906906
# rendered 'MD PhD' since 1.4.0 (#429). Ask assign's own
907907
# predicate, over the pieces group just built.
908+
# `family_comma` is redundant by invariant and kept for
909+
# locality: segment() emits at most one segment for NO_COMMA, so
910+
# seg_idx == 1 already implies a comma, and under SUFFIX_COMMA
911+
# tail_start is 1, so `tail` short-circuits before this. Nothing
912+
# can pin it -- dropping it is an equivalent mutant over the
913+
# corpora and 65,725 generated inputs -- so it is documented
914+
# rather than tested.
908915
one_entry = tail or (
909916
family_comma and seg_idx == 1
910917
and _segment_holds_no_name(pieces, ptags, tokens))
@@ -917,7 +924,7 @@ def group(state: ParseState) -> ParseState:
917924
# tokens within an entry take the stable "joined" tag so
918925
# the suffix view space-joins them (the fix_phd mechanism).
919926
# Core dropping stays keyed on `tail` via seg_cores: the
920-
# #191 parity is a TAIL rule, and the one-entry join is the
927+
# #206 parity is a TAIL rule, and the one-entry join is the
921928
# only half that follows assign's content read.
922929
entry_open = False
923930
kept: list[int] = []
@@ -932,8 +939,11 @@ def group(state: ParseState) -> ParseState:
932939
kept.append(k)
933940
# Two different joins, and conflating them is what a
934941
# widened condition gets wrong. WITHIN a piece (pos > 0)
935-
# the tag renders a merged piece as one unit, whatever
936-
# role it holds. BETWEEN pieces it continues an ENTRY,
942+
# the tag renders a merged piece as one unit; the branch
943+
# is written role-blind because the merge is (the ph-d
944+
# pair reaches GIVEN as one element), though no
945+
# multi-token TITLE piece witnesses it -- none turned up
946+
# in 38,892 generated family-comma inputs. BETWEEN pieces it continues an ENTRY,
937947
# and only pieces that render into the same run may do
938948
# that. On a tail segment every kept piece does -- that
939949
# is what `tail` means -- but off it assign routes piece

tests/v2/cases.py

Lines changed: 1 addition & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -582,7 +582,7 @@ def __post_init__(self) -> None:
582582
Case("suffix_delimiter_tail_segment", "Doe, John, RN - CRNA",
583583
{"given": "John", "family": "Doe", "suffix": "RN, CRNA"},
584584
policy=_SD,
585-
notes="v1 suffix_delimiter parity (#191): the delimiter token "
585+
notes="v1 suffix_delimiter parity (#206): the delimiter token "
586586
"is dropped from consumed tail segments (pinned live "
587587
"2026-07-16)"),
588588
Case("suffix_delimiter_detection", "Doe, John RN - CRNA",
@@ -1647,24 +1647,6 @@ def __post_init__(self) -> None:
16471647
"'Jr. Jr.', the exact inverse of the bug #429 fixes. "
16481648
"Only a piece that renders into the same run may "
16491649
"continue an entry"),
1650-
Case("family_comma_title_run_does_not_join", "Smith, Rev. Dr.",
1651-
{"title": "Rev. Dr.", "family": "Smith"},
1652-
notes="the same regression seen through the other view: the "
1653-
"'joined' tag is role-blind and the facade heals it for "
1654-
"EVERY role, so a title piece opening an entry "
1655-
"collapsed title_list from ['Rev.', 'Dr.'] to "
1656-
"['Rev. Dr.']. The title string is space-joined either "
1657-
"way, so THIS row cannot see the collapse and does not "
1658-
"guard it -- test_facade.py's "
1659-
"test_the_joined_tag_never_reaches_a_title asserts the "
1660-
"list views and does. Kept here for the shape; the "
1661-
"differential compares strings and is blind to the "
1662-
"whole class"),
1663-
Case("family_comma_three_credential_entries", "Smith, Ph. D. Jr. MD",
1664-
{"family": "Smith", "suffix": "Ph. D. Jr. MD"},
1665-
classification="fix(#429)",
1666-
notes="the run does not stop at two: the entry latches rather "
1667-
"than being reset per piece"),
16681650
Case("family_comma_written_commas_are_kept", "Smith, MD, PhD",
16691651
{"family": "Smith", "suffix": "MD, PhD"},
16701652
notes="the negative control for #429, and the distinction the "

tests/v2/pipeline/test_group.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -436,7 +436,7 @@ def test_the_connective_carveout_counts_the_surviving_name() -> None:
436436

437437

438438
def test_a_delimiter_core_in_a_suffix_tail_is_not_maiden_text() -> None:
439-
"""A tail segment drops its delimiter cores (#191) and the marker
439+
"""A tail segment drops its delimiter cores (#206) and the marker
440440
takes what is left, in that order -- the order group() had before
441441
the marker pass moved ahead of the joins. A core is not a word the
442442
marker can take: it is structure, like the marker itself."""

tests/v2/test_facade.py

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -207,6 +207,17 @@ def test_the_joined_tag_never_reaches_a_title( # #429 regression guard
207207
k = HumanName("Smith, MD PhD")
208208
assert k.suffix_list == ["MD PhD"] # ONE element
209209

210+
# And the pin for the TAG CONDITION itself, which the three above
211+
# miss: they all pin the sticky entry_open update, and the whole
212+
# suite stays green with `in_entry and` dropped from the tag test.
213+
# This needs a suffix piece FIRST -- opening the entry legitimately
214+
# -- and then TWO titles, so the second title is a continuation of
215+
# an entry it does not belong to.
216+
j = HumanName("Smith, MD Rev. Dr.")
217+
assert j.suffix == "MD"
218+
assert j.title == "Rev. Dr." # identical either way
219+
assert j.title_list == ["Rev.", "Dr."] # the mutant gives ['Rev. Dr.']
220+
210221

211222
def test_str_uses_string_format_with_v1_cleanup() -> None:
212223
n = HumanName("Dr. Juan de la Vega III")

0 commit comments

Comments
 (0)