buchwandler

API reference

Preparation

.. py:function:: prepare(text, *, language=’en’, config=None, profile=None, annotations=None, nlp=None, protected_spans=None, lexical_evidence=None, use_spacy=None, spacy_model=None, expand_abbreviations=True, expand_structured=True, normalize_literals=False, expand_numbers=True, normalize_whitespace=True, normalize_unicode=True, strip_outer_whitespace=True, collapse_horizontal_whitespace=True, normalize_line_whitespace=True, collapse_blank_lines=True, number_policy=None, preserve_run_boundaries=False, model_punctuation=False, symbol_mode=’none’, keep_symbols=’’, generic_acronym_mode=’known_only’, generic_acronym_case=’upper’, long_number_mode=’preserve’, registered_acronym_mode=’expand’, context=True, interpretation_mode=InterpretationMode.CONTEXTUAL, disabled_domains=frozenset({}), allowed_domains=None, sequence_fallback_mode=SequenceFallbackMode.PRESERVE, strict=False) :module: spokenform

Convert one-language written text into a readable form intended for speech.

The caller selects the processing language. Language detection, mixed-language segmentation, and markup parsing belong outside spokenform.

Structured values run before lexical abbreviation expansion and generic numbers so each complete expression receives one semantic replacement.

.. py:function:: prepare_language(text, *, language, **kwargs) :module: spokenform

Prepare one explicitly selected language run with generic policy.

Unlike :func:prepare, this future-facing entry point intentionally has no compatibility default for language. The optional config must use the same language so the call cannot silently process a different language.

.. py:function:: prepare_for_kokorog2p(text, language=’en’, *, config=None, profile=None, **kwargs) :module: spokenform

Prepare one language with the kokorog2p-safe profile.

.. py:function:: prepare_for_piperg2p(text, language=’en’, *, config=None, profile=None, **kwargs) :module: spokenform

Prepare one language with the PiperG2P-safe profile.

.. py:function:: normalize_spacing(text, *, normalize_unicode=True, strip_outer_whitespace=True, collapse_horizontal_whitespace=True, normalize_line_whitespace=True, collapse_blank_lines=True, number_policy=None) :module: spokenform

Apply independently configurable Unicode and whitespace policies.

number_policy remains accepted for 0.2.x compatibility. Numeric policy selection belongs to :func:prepare; spacing itself does not consume it.

normalize_unicode, strip_outer_whitespace, collapse_horizontal_whitespace, normalize_line_whitespace, and collapse_blank_lines are independent policy controls. normalize_whitespace=False remains the compatibility switch for skipping the whitespace stage entirely. normalize_literals=True opts into high-confidence URL, e-mail, semantic-version, and contextual Roman rendering; caller-protected spans always take precedence.

Language identifiers and number backends

Canonical runtime identifiers include 49 base families and 18 exact public locale overlays. supported_languages() returns the base families; supported_languages(include_locales=True) returns all 67 public keys. Exact overlays are preserved for abbreviation routing, while unregistered regional forms fall back to their base language. kk is Spokenform’s Kazakh key, maps to kz for abbr2words, and maps to kk for Numeralform.

The installed Numeralform renderer is capability-based. Chinese uses cn2an; Numeralform availability is reported separately from automatic plain-cardinal ownership, so hi, hy, and mn remain valid language inputs while preserving ordinary numeric source until their normalization ownership is reviewed. Reviewed decimal policies, structured locale grammar, sequence spelling, and KokoroG2P profiles are narrower capabilities and are not implied by global language support.

from spokenform import (
    language_support,
    normalize_language,
    resolve_abbr2words_language,
    resolve_numeralform_locale,
    supported_languages,
    supports_language,
)

assert len(supported_languages()) == 49
assert len(supported_languages(include_locales=True)) == 66
assert "nl" in supported_languages()
assert normalize_language("pt-BR") == "pt_BR"
assert normalize_language("fr_FR") == "fr"
assert normalize_language("en-gb") == "en_GB"
assert resolve_abbr2words_language("es-ni") == "es_NI"
assert resolve_numeralform_locale("fr-FR") == "fr"
assert language_support("hi").plain_cardinals is False
assert supports_language("eu") is False

symbol_mode="none" is the backward-compatible default and applies no general residual-symbol filter. symbol_mode="remove" removes Unicode punctuation and symbol characters (P* and S*) left after semantic recognition. With symbol_mode="keep", keep_symbols is an exact-codepoint allowlist; it must not be empty. An allowlist is invalid in none or remove mode. The active filter is recorded as a symbols stage before whitespace normalization and never changes protected spans. Symbol deletions are included in stage and source/output mapping.

generic_acronym_case="upper" is the default. Set it to "lower" to lowercase only generic grapheme-spaced uppercase acronyms; lexical acronyms, preserved terms, known initialisms, identifiers, and mixed-case tokens keep their normal policies.

Configuration policy modes

The policy modes below are passed through the selected language’s abbr2words registry where abbreviation or initialism data is involved. They are available for every language profile, but the exact recognized vocabulary and number grammar remain locale-specific. They are safety controls: conservative modes prefer leaving an ambiguous token unchanged, while expansive modes can create false positives in identifiers, product names, headlines, years, and other uppercase or digit-heavy text.

Generic acronyms

generic_acronym_mode="known_only" is the default. It lets registered or otherwise known initialisms follow their registry policy and leaves unknown uppercase words alone. For example, an unknown TST remains unchanged while a known entry such as ABC can be rendered according to the registry.

"conservative_unknown" additionally spells unknown uppercase initialisms when the context provides abbreviation evidence, reducing accidental changes to ordinary uppercase prose. "spell_unknown" is the broadest mode: eligible unknown uppercase initialisms are spelled even without as much contextual evidence, so it is useful for abbreviation-heavy input but has the highest false-positive risk.

PreparationConfig(language="en", generic_acronym_mode="known_only")
PreparationConfig(
    language="en",
    generic_acronym_mode="spell_unknown",
    generic_acronym_case="upper",
)

For example, with use_spacy=False, prepare("ABC TST", language="en", generic_acronym_mode="conservative_unknown") produces "A B C T S T" in the current English registry, while the default keeps the unknown TST unchanged. generic_acronym_case="lower" changes only generic grapheme-spaced output ("a b c t s t" in that example); it does not lowercase lexical acronyms or registered terms. These modes are delegated to abbr2words’ initialism policy rather than implemented as a second local abbreviation registry.

Registered acronyms

registered_acronym_mode="expand" is the default and uses the registered initialism expansion supplied by abbr2words. Set it to "spell" when the letters of registered initialisms are preferable to their lexical expansion, for example CEO MIT -> C E O M I T in English. This mode has lower lexical interpretation risk but may be less natural for acronyms whose full names are well known. It affects registered entries, not arbitrary unknown uppercase words, and is forwarded to abbr2words as registered_initialism_mode.

PreparationConfig(language="en", registered_acronym_mode="spell")

Long numbers

long_number_mode="preserve" is the default. It leaves long ungrouped digit strings alone when they could be years, identifiers, account numbers, or other downstream-owned sequences. "contextual" verbalizes a long number when quantity evidence makes cardinal reading safe, such as 844361 items, while continuing to protect identifier-like uses. "cardinal" requests cardinal verbalization for eligible long numbers regardless of that contextual evidence; it is useful for prose known to contain quantities but carries a higher risk of rewriting IDs and years. The number grammar and rendered words are selected by the active language, and this policy is independent of abbr2words’ lexical abbreviation registry.

PreparationConfig(language="en", long_number_mode="contextual")

For English, prepare("844361 items", language="en", long_number_mode="preserve") keeps the digits, while "contextual" and "cardinal" produce "eight hundred forty four thousand three hundred sixty one items".

.. py:class:: PreparationConfig(language=’en’, use_spacy=None, spacy_model=None, expand_abbreviations=True, expand_structured=True, normalize_literals=False, expand_numbers=True, normalize_whitespace=True, normalize_unicode=True, strip_outer_whitespace=True, collapse_horizontal_whitespace=True, normalize_line_whitespace=True, collapse_blank_lines=True, number_policy=None, preserve_run_boundaries=False, model_punctuation=False, symbol_mode=’none’, keep_symbols=’’, generic_acronym_mode=’known_only’, generic_acronym_case=’upper’, long_number_mode=’preserve’, registered_acronym_mode=’expand’, interpretation_mode=InterpretationMode.CONTEXTUAL, sequence_fallback_mode=SequenceFallbackMode.PRESERVE, disabled_domains=frozenset({}), allowed_domains=None, context=True, strict=False) :module: spokenform :canonical: spokenform.config.PreparationConfig

Immutable options controlling single-language written-to-spoken preparation.

.. py:method:: PreparationConfig.for_kokorog2p(language) :module: spokenform :classmethod:

  Return a one-language profile safe for kokorog2p adapters.

.. py:method:: PreparationConfig.for_piperg2p(language) :module: spokenform :classmethod:

  Return a one-language profile safe for piperg2p adapters.

.. py:method:: PreparationConfig.for_speech(language) :module: spokenform :classmethod:

  Return the generic one-language speech preparation preset.

Result models

.. py:class:: PreparedText(source_text, clean_text, spoken_text, language, stages=(), mapped_edits=(), source_replacements=(), protected_spans=(), reserved_spans=(), offset_map=None, warnings=()) :module: spokenform :canonical: spokenform.models.PreparedText

Readable spoken text with normalization provenance.

The mapping describes coordinates only. It does not promise that source linguistic annotations remain valid for generated tokens in spoken_text. Downstream POS or morphology analysis must run after preparation.

.. py:property:: PreparedText.changed :module: spokenform :type: bool

  Return whether preparation changed the source text.

.. py:property:: PreparedText.edits :module: spokenform :type: tuple[~spokenform.models.TextEdit, …]

  Return the ordered edits from every stage.

.. py:method:: PreparedText.map_output_span(start, end) :module: spokenform

  Map a final spoken-text span back to original source coordinates.

.. py:method:: PreparedText.map_source_span(start, end) :module: spokenform

  Map an original source span to final spoken-text coordinates.

.. py:method:: PreparedText.render_changes() :module: spokenform

  Render a compact, human-readable stage report.

.. py:property:: PreparedText.replacements :module: spokenform :type: tuple[~spokenform.models.SourceReplacement, …]

  Stable adapter alias for composed source replacements.

.. py:property:: PreparedText.source_edits :module: spokenform :type: tuple[~spokenform.models.SourceReplacement, …]

  Return edits in the documented source-to-final coordinate space.

.. py:property:: PreparedText.stage_report :module: spokenform :type: str

  Return the diagnostic stage report for adapter logging.

.. py:property:: PreparedText.text :module: spokenform :type: str

  Alias for :attr:`spoken_text`.

.. py:method:: PreparedText.to_adapter_dict() :module: spokenform

  Return the stable downstream-adapter result projection.

.. py:method:: PreparedText.to_dict() :module: spokenform

  Return a JSON-serializable representation.

All public source offsets refer to the original string passed to prepare(); final offsets refer to PreparedText.spoken_text; stage-local offsets remain available only under each PreparationStage. PreparedText.to_adapter_dict() is the stable JSON-ready projection for downstream adapters.

prepare_for_kokorog2p() requires an explicit language and performs no language detection, tokenization, G2P, or model-punctuation rewriting. Its default profile preserves run boundary whitespace. Pass caller-owned protected_spans to prevent semantic replacements; partial overlap is handled fail-closed by protecting the complete recognized quantity expression. Use source_replacements and the offset helpers to rebase downstream token and override coordinates.

PreparationConfig.for_speech(language) provides the generic one-language speech preset. It contains no TTS-engine policy. PreparationConfig.for_kokorog2p() and PreparationConfig.for_piperg2p() produce compatibility configurations for their downstream adapters and are not the architectural center of Spokenform.

PreparedText mappings describe source and output coordinates, not a linguistic analysis of generated tokens. When normalization changes a token, source POS, tag, lemma, or morphology must not be reused for the generated text. Run a fresh linguistic analysis after preparation when downstream processing requires it. The preferred application surface is prepare_language() with PreparationConfig.for_speech(language), PreparedText.source_replacements, and PreparedText.offset_map. The KokoroG2P compatibility surface additionally uses PreparationConfig.for_kokorog2p(), prepare_for_kokorog2p(), and NumberPolicy. Lower-level mapping and stage helpers are advanced exports rather than requirements for a normal adapter.

prepare_for_piperg2p() uses the same prepared-text boundary with Spokenform’s reviewed generic number ownership. It performs no language detection, Piper configuration loading, G2P, phonemization, or model-ID encoding. The caller passes PreparedText.spoken_text and keeps the semantic Spokenform language separate from the Piper voice identifier such as en-us. PiperG2P remains a separately installed downstream package.

KokoroG2P retains its legacy handoff numeric policy. PiperG2P consumes already-prepared text and therefore uses the current generic Spokenform numeric ownership policy. Both profiles preserve source replacements, offset mapping, protected spans, and run boundaries.

The stable application-facing surface is prepare_language(), prepare(), prepare_for_kokorog2p(), prepare_for_piperg2p(), prepare_text, PreparationConfig, NumberPolicy, PreparedText, ProtectedSpan, ProtectionError, TokenAnnotation, and __version__. number_policy_for_language() and normalize_numbers() are stable locale-policy helpers. The annotation adapters, spaCy model helpers, structured-stage helpers, StageResult, and mapping/replacement classes and conversion functions are advanced public APIs: they remain exported for compatibility and diagnostics, but downstream integrations should prefer the high-level preparation surface. No exported symbol is removed in 0.2.2.

.. py:class:: PreparationStage(name, before, after, edits=(), mapped_edits=(), reserved=()) :module: spokenform :canonical: spokenform.models.PreparationStage

The before/after text and edits produced by one stage.

.. py:property:: PreparationStage.changed :module: spokenform :type: bool

  Return whether this stage changed the text.

.. py:class:: TextEdit(start, end, source, replacement, stage) :module: spokenform :canonical: spokenform.models.TextEdit

One edit in a normalization stage.

Offsets are relative to the input of that stage. This keeps every edit exact without pretending that all stages still share the original coordinate space.

.. py:class:: MappedEdit(source_start, source_end, output_start, output_end, source, replacement, stage, language=None, kind=’replacement’, rule=None, recognition_domain=None, recognition_evidence=None, evidence_source=None, evidence_score=None, evidence_cues=()) :module: spokenform :canonical: spokenform.models.MappedEdit

A replacement with source and output coordinates.

Coordinates and semantic metadata describe the generated text. Source linguistic annotations are not carried as a valid analysis of replacement text; downstream POS or morphology consumers must analyze the final text.

Annotation adapters

.. py:class:: TokenAnnotation(start, end, text=None, pos=None, tag=None, lemma=None, language=None) :module: spokenform :canonical: spokenform.models.TokenAnnotation

Provider-neutral lexical annotation aligned to one input text.

.. py:function:: annotations_from_spacy(doc) :module: spokenform

Convert a spaCy-like Doc into source-aligned annotations.

The adapter imports no spaCy modules and can therefore be used with compatible providers or simple test doubles.

.. py:function:: spacy_annotations(text, nlp) :module: spokenform

Run an existing spaCy-compatible pipeline and convert its tokens.

.. py:function:: validate_annotations(text, annotations) :module: spokenform

Validate and materialize source-aligned annotations.

Annotation spans must be ordered, non-overlapping, inside text, and match annotation.text when that optional value is supplied.

Number normalization

The Czech normalize_numbers(language="cs") path delegates to the reviewed structured and structured-safe plain-number grammar. It verbalizes ordinary numbers, validated dates, quantities, temperatures, and canonical currencies; colon-time candidates remain unchanged for caller-managed handling.

.. py:function:: normalize_numbers(text, *, language) :module: spokenform

Verbalize common dates, times, currencies, ordinals, and numbers.

URLs, email addresses, and semantic-version-like values are protected. The implementation is intentionally conservative and is an MVP, not a complete locale grammar.

.. py:function:: normalize_structured(text, *, language, protected_ranges=(), promote_literals=False, generic_acronym_mode=’known_only’, generic_acronym_case=’upper’, interpretation_mode=InterpretationMode.CONTEXTUAL, disabled_domains=frozenset({}), allowed_domains=None, evidence=None, trace=None) :module: spokenform

Normalize structured values and return exact semantic provenance.

.. py:function:: iter_structured_replacements(text, *, language, protected_ranges=(), promote_literals=False, generic_acronym_mode=’known_only’, generic_acronym_case=’upper’, interpretation_mode=InterpretationMode.CONTEXTUAL, disabled_domains=frozenset({}), allowed_domains=None, evidence=None, trace=None) :module: spokenform

Return exact, non-overlapping semantic replacements for one language.

Interpretation policy

PreparationConfig and prepare() expose two orthogonal recognition controls:

  • interpretation_mode="contextual" (default) permits reviewed contextual semantic evidence.

  • interpretation_mode="surface" permits only candidates marked with intrinsic evidence and fails closed on missing metadata.

  • disabled_domains={"chemistry", ...} suppresses selected RecognitionDomain families before precedence resolution.

  • allowed_domains={"quantities", ...} permits only selected semantic families and fails closed for candidates without domain metadata. If a domain appears in both sets, configuration raises ValueError.

from spokenform import prepare_language

prepare_language(
    "The sample contains H2O.",
    language="en",
    interpretation_mode="surface",
    disabled_domains={"chemistry"},
)

context remains the legacy abbreviation-context switch. Surface mode clamps its effective abbreviation context off, but context=False under contextual mode does not disable structured recognizers. InterpretationMode, RecognitionDomain, and RecognitionEvidence are exported public types. Policy-suppressed candidates are visible in structured trace diagnostics.

sequence_fallback_mode="preserve" is the compatibility default. Set it to "spell" to render conservative residual sequence-shaped spans such as AAPL or H2O orthographically after semantic recognition. It does not spell ordinary lexical prose, does not claim a semantic domain, and never overrides caller-protected or auto-protected literal spans. SequenceFallbackMode, InterpretationMode, RecognitionDomain, and RecognitionEvidence are exported public types. Policy-suppressed candidates are visible in structured trace diagnostics.

The Swedish normalize_numbers(language="sv") path preserves comma-decimal precision and valid numeric grouping, while dates, digital times, and ambiguous dot forms remain caller-managed. Swedish structured quantities, temperatures, and SEK currency are owned by the locale grammar used by prepare().

The Vietnamese normalize_numbers(language="vi") path uses structured-safe normalization. Comma decimals preserve exact fractional precision, dot and space-family grouping are validated, and reviewed quantities and VND/₫ are locale-owned through abbr2words. Date, time, and ordinal semantics remain caller-managed.

Thai normalize_numbers(language="th") uses point decimals, comma or space-family grouping, accepts Thai digits, and preserves decimal precision digitwise. Structured quantities, temperatures, and THB/฿ use reviewed abbr2words identities while Numeralform owns numeric realization. th_TH and th-TH resolve to the Numeralform base locale; dates, times, eras, ordinals, ranges, and unsupported specialist semantics remain caller-managed or fail closed.

Russian normalize_numbers(language="ru") uses the structured-safe path. It preserves comma-decimal precision, accepts regular/NBSP/NNBSP grouping, and renders reviewed quantity grammar through abbr2words canonical IDs. Dates, digital times, year abbreviations, phone spans, RUB, and unreviewed specialist semantics remain caller-managed or fail closed. rus is accepted only as a compatibility alias.

Lexhint evidence provider

Lexhint 0.1.2 <= x < 0.3.0 is supported as an optional provider. Lexhint 0.1.x uses schema-7 artifacts, while Lexhint 0.2.x requires schema-8 artifacts installed separately with lexhint dataset download <language> --variant runtime. Spokenform never downloads datasets automatically.

Pass an installed Lexhint runtime Lexicon explicitly through lexical_evidence. No dataset is resolved or downloaded when the argument is omitted. URL lexical segmentation may work with a lexical-only provider. Contextual computing and sports support requires the semantic capability, and missing semantic evidence never vetoes an existing candidate.

from lexhint import Lexicon
from spokenform import prepare

lexicon = Lexicon("en", variant="runtime")
result = prepare(
    "compiler 8.3.2 and chatgpt.com",
    language="en",
    normalize_literals=True,
    lexical_evidence=lexicon,
    use_spacy=False,
)

Lexhint semantic domains are mapped to Spokenform use cases rather than copied into RecognitionDomain. Surface mode ignores semantic evidence. URL lexical evidence is a rendering aid for a URL already recognized or promoted by Spokenform.