Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/concepts.rst
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,8 @@ reading is a fact about the domain the names come from: that makes it a
square brackets — ``John [Johnny] Smith`` — instead of quotes is a
fact about that one data source's export format, not about the
language of the names in it — that's a
:class:`~nameparser.Policy` (``nickname_delimiters={('[', ']')}``).
:class:`~nameparser.Policy`
(``nickname_delimiters=frozenset({('[', ']')})``).
One particular report
wanting names formatted as "Family, Given" while every other consumer
of the same parsed data wants "Given Family" is a fact about where the
Expand Down
31 changes: 20 additions & 11 deletions docs/customize.rst
Original file line number Diff line number Diff line change
Expand Up @@ -335,10 +335,10 @@ this section is how to switch them off, which you can do separately:

>>> parse("김민준").family # both defaults on
'김'
>>> positional = Parser(policy=Policy(script_orders={}))
>>> positional = Parser(policy=Policy(script_orders=()))
>>> positional.parse("김민준").family # still split
'민준'
>>> unsplit = Parser(policy=Policy(segment_scripts=()))
>>> unsplit = Parser(policy=Policy(segment_scripts=frozenset()))
>>> unsplit.parse("김민준").family # one token, not split
'김민준'

Expand All @@ -363,8 +363,9 @@ because splitting Han text requires knowing Chinese from Japanese;
:doc:`locales` covers the opt-in ``zh`` pack that supplies them.

The Japanese behaviors ride these same two fields, so they need no
switches of their own: ``script_orders={}`` clears the kana-licensed
entry along with the Han and Hangul ones, and ``segment_scripts=()``
switches of their own: ``script_orders=()`` clears the kana-licensed
entry along with the Han and Hangul ones, and
``segment_scripts=frozenset()``
deactivates every script at once, which also stops a parser consulting
whatever segmenter it was given. The segmenter has an off-switch as
well — ``Parser(segmenter=None)``, which is the default; see
Expand All @@ -389,13 +390,21 @@ off.

.. note::

Both fields are annotated with their canonical *storage* type
Every field here is annotated with its canonical *storage* type
rather than with everything the constructor accepts — the same as
``capitalization_exceptions``. Under mypy the readable spellings
above (``script_orders={...}``, ``segment_scripts=(...)``) need a
``# type: ignore[arg-type]``; ``script_orders=()`` and
``segment_scripts=frozenset(...)`` check clean and mean the same
thing.
``capitalization_exceptions``, and for the same reason: the
annotation is what you get back when you READ the attribute, which
is the commoner operation.

The constructor is deliberately wider. It takes any mapping for
``script_orders``, any iterable of ``Script`` for
``segment_scripts``, and plain strings wherever a ``Role`` is
wanted (``Role`` is a ``StrEnum`` precisely so that works). A
dataclass cannot express those two types separately, so the
examples in this guide use the spellings that check clean under
mypy — ``()`` and ``frozenset(...)`` rather than ``{}`` and a bare
set literal. The wider spellings parse identically; they just need
a ``# type: ignore[arg-type]`` if you run a type checker.

Nicknames, maiden names, and brackets
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Expand All @@ -407,7 +416,7 @@ one-liner is the whole recipe:

.. doctest::

>>> policy = Policy(maiden_delimiters={("(", ")")})
>>> policy = Policy(maiden_delimiters=frozenset({("(", ")")}))
>>> Parser(policy=policy).parse("Jane (Jones) Smith").maiden
'Jones'

Expand Down
2 changes: 1 addition & 1 deletion docs/migrate.rst
Original file line number Diff line number Diff line change
Expand Up @@ -427,7 +427,7 @@ now.

``Constants`` has no switch for any of this — the v1 configuration
surface is frozen for 2.x — so the way out is the 2.0 API:
``Parser(policy=Policy(script_orders={}, segment_scripts=()))``
``Parser(policy=Policy(script_orders=(), segment_scripts=frozenset()))``
restores 1.4's reading of every shape above that turns on order or
splitting. The middle dots are the exception: both the katakana dot
and the Chinese interpunct ``·`` (U+00B7, dividing a transcription
Expand Down
7 changes: 4 additions & 3 deletions docs/modules.rst
Original file line number Diff line number Diff line change
Expand Up @@ -140,9 +140,10 @@ because only these three orders have defined assignment semantics.
the names no entry matches. The values are drawn from the same
three constants above, and the same restriction applies. Build on it for
additive customization —
``script_orders=dict(DEFAULT_SCRIPT_ORDERS) | {Script.HAN:
GIVEN_FIRST}`` — and pass ``script_orders={}`` to opt out entirely
and get the purely positional read back. Latin-script and
``script_orders=(*DEFAULT_SCRIPT_ORDERS, (Script.HAN, GIVEN_FIRST))``,
where a later entry for a script REPLACES an earlier one, so
appending is how you override — and pass ``script_orders=()`` to
opt out entirely and get the purely positional read back. Latin-script and
mixed-script names are never affected either way.

Delimiter defaults
Expand Down
25 changes: 25 additions & 0 deletions tests/v2/test_policy.py
Original file line number Diff line number Diff line change
Expand Up @@ -565,6 +565,31 @@ def test_script_orders_default_and_canonical_storage() -> None:
assert Policy(script_orders={}).script_orders == () # type: ignore[arg-type]


def test_a_later_script_orders_entry_replaces_an_earlier_one() -> None:
"""Appending is how you override a single script, and the docs now
say so (modules.rst, the script_orders entry).

The behavior falls out of _validated_script_orders building a dict
before sorting, so it was emergent rather than contracted -- and
prose that tells readers to append is prose that depends on it.
The type-clean spelling for "everything as shipped, but Han reads
given-first" is exactly this, which is why it is worth pinning.
"""
overridden = Policy(
script_orders=(*DEFAULT_SCRIPT_ORDERS, (Script.HAN, GIVEN_FIRST)))
assert dict(overridden.script_orders)[Script.HAN] == GIVEN_FIRST
# the other shipped entries survive untouched
assert dict(overridden.script_orders)[Script.HANGUL] == FAMILY_FIRST
assert dict(overridden.script_orders)[Script.HIRAGANA] == FAMILY_FIRST
# one entry per script, whatever the input carried
keys = [s for s, _ in overridden.script_orders]
assert len(keys) == len(set(keys)) == len(DEFAULT_SCRIPT_ORDERS)
# and the mapping spelling it replaces still means the same thing
assert overridden == Policy(
script_orders=dict(DEFAULT_SCRIPT_ORDERS) # type: ignore[arg-type]
| {Script.HAN: GIVEN_FIRST})


def test_script_orders_validates_keys_and_values() -> None:
with pytest.raises(ValueError, match="han, hangul"):
Policy(script_orders={"klingon": FAMILY_FIRST}) # type: ignore[arg-type]
Expand Down