Core API
This module contains the core functionality of kokorog2p.
Main Functions
Structured stress overrides
OverrideSpan supports explicit phoneme stress changes through the stress attribute.
Values are strings and must be one of "-2", "-1", "+1", or "+2". The stress
change is applied after language resolution and pronunciation or ph override
resolution.
from kokorog2p import OverrideSpan, phonemize
result = phonemize(
"zwei Minuten",
language="de",
overrides=[OverrideSpan(0, 4, {"stress": "+2"})],
)
The values mean:
-2: remove stress markers.-1: lower stress by one level.+1: raise stress by one level.+2: promote or force primary stress.
A span covering multiple lexical tokens applies independently to each spoken token. This
changes phoneme stress markers (ˌ and ˈ), not volume, pitch, duration, or SSML
emphasis. Markdown, Misaki, Kokoro markup, and other prosody syntax are not supported.
Prepared input
Use phonemize_prepared when the caller has already converted written text to spoken
text. It retains kokorog2p tokenization, G2P, model punctuation handling, overrides,
phoneme construction, and token IDs, but does not run written-to-spoken semantic
expansion again.
All canonical public entry points require the language to be supplied explicitly.
phonemize, phonemize_prepared, and tokenize require language=...; the
convenience wrappers phonemes and phoneme_ids follow the same rule.
- kokorog2p.phonemize_prepared(text: str, language: str, overrides: Sequence[OverrideSpanLike] | None = None, annotations: Sequence[TokenAnnotationLike] | None = None, return_ids: bool = True, return_phonemes: bool = True, alignment: Literal['span', 'legacy'] = 'span', overlap: Literal['snap', 'strict', 'split'] = 'snap', use_normalizer_rules: bool = True, use_espeak_fallback: bool = True, use_goruut_fallback: bool = False, use_cli: bool = False, use_spacy: bool | None = None, spacy_model: str | None = None, spacy_model_size: SpacyModelSize | None = None, lexicons: str | Sequence[str] | None = None, backend: Literal['kokorog2p', 'espeak', 'goruut'] = 'kokorog2p', g2p: G2PBase | None = None, g2p_resolver: Callable[[str], G2PBase] | None = None, language_routing: LanguageRoutingConfig | Mapping[str, Any] | None = None, target_model: str | None = None, g2p_options: Mapping[str, Any] | None = None, strict_stress: bool = False) PhonemizeResult[source]
Explicit alias for the prepared-text phonemization contract.
The caller owns written-to-spoken semantic preparation; kokorog2p performs tokenization, overrides, G2P, model punctuation handling, and ID generation.
textis the prepared coordinate space, so override offsets and returned token offsets refer directly to it.
- kokorog2p.phonemize(text: str, language: str, overrides: Sequence[OverrideSpanLike] | None = None, annotations: Sequence[TokenAnnotationLike] | None = None, return_ids: bool = True, return_phonemes: bool = True, alignment: Literal['span', 'legacy'] = 'span', overlap: Literal['snap', 'strict'] = 'snap', use_normalizer_rules: bool = True, use_espeak_fallback: bool = True, use_goruut_fallback: bool = False, use_cli: bool = False, use_spacy: bool | None = None, spacy_model: str | None = None, spacy_model_size: SpacyModelSize | None = None, lexicons: str | Sequence[str] | None = None, backend: Literal['kokorog2p', 'espeak', 'goruut'] = 'kokorog2p', g2p: G2PBase | None = None, g2p_resolver: Callable[[str], G2PBase] | None = None, language_routing: LanguageRoutingConfig | Mapping[str, Any] | None = None, target_model: str | None = None, g2p_options: Mapping[str, Any] | None = None, strict_stress: bool = False) PhonemizeResult[source]
Phonemize text using the unified kokorog2p pipeline.
This is the primary public entry point for turning text into phonemes (and optionally tokens or token IDs) in a consistent way.
Internally, this function delegates to the same implementation used by span-based override phonemization (the former
phonemize_to_resultpath), ensuring that:The phoneme string returned here is identical to the one in the returned
PhonemizeResult.Tokenization and character offsets are deterministic and match the phoneme output.
Kokoro-model vocabulary validation/filtering is applied when producing token IDs (and when necessary to make the phoneme string ID-safe).
- Args:
- text:
Input text to phonemize. This should be plain text (no markup). Punctuation may be normalized (e.g.
...→…,-→—) to match Kokoro-compatible forms.- language:
Language code (e.g.
"en-us","en-gb","de","fr"). Used both for tokenization/alignment and for constructing a default G2P instance wheng2pis not provided.- overrides:
Optional span-based overrides applied by character offsets. Overrides can inject phonemes (
{"ph": "…"}) and/or change the language of a span ({"lang": "de"}) for that region.- return_ids:
Whether to include token IDs in the returned result.
- return_phonemes:
Whether to include the phoneme string in the returned result.
- alignment:
Alignment mode for applying overrides and token offsets:
"span"(default): deterministic offset-based alignment usingtokenize()."legacy": backward-compatible alignment based on the backend’s own tokenization. This may differ slightly across backends and languages.
- overlap:
How to handle overrides that partially overlap a token boundary:
"snap"(default): apply to intersecting tokens and emit a warning when boundaries only partially overlap."strict": skip partial overlaps and emit a warning.
- use_normalizer_rules:
Whether to apply language normalizer rules when building the internal alignment text used for span mapping.
- use_espeak_fallback:
When constructing a G2P instance for the dictionary-based
"kokorog2p"backend, fall back to eSpeak for out-of-vocabulary words. Ignored ifg2pis provided.- use_goruut_fallback:
When constructing a G2P instance for the dictionary-based
"kokorog2p"backend, fall back to goru·ut for out-of-vocabulary words. Ignored ifg2pis provided.- use_spacy:
When constructing a G2P instance, whether to use spaCy for tokenization/POS tagging (English/French and optionally German). For Chinese/Japanese/Korean, this flag is accepted for API consistency but currently does not alter backend behavior. Ignored if
g2pis provided.- spacy_model:
Concrete spaCy model package or
"auto"when constructing a G2P instance. If omitted, the highest installed loadable model is used when spaCy is enabled. For Chinese/Japanese/Korean, accepted for API consistency but not currently used by native backends.- spacy_model_size:
Exact spaCy model tier to use when
spacy_modelis omitted.- lexicons:
A named lexicon or ordered sequence of named lexicons for the language. When omitted, the language default stack is used. In an explicit sequence, the first lexicon wins collisions. Use
available_lexicons(language)to inspect registered names.- backend:
When constructing a G2P instance, select the backend:
"kokorog2p","espeak", or"goruut". Ignored ifg2pis provided.- g2p:
Optional pre-created G2P instance to reuse across calls (useful for caching/performance). If provided, this function will use it directly and will NOT call
get_g2p()(sobackendand the fallback/spaCy construction flags are ignored for this call).
- Returns:
A
PhonemizeResultcontaining tokens, phonemes, token_ids, and warnings (depending onreturn_*flags).- Examples:
Basic phonemization:
>>> phonemize("Hello world!", language="en-us").phonemes 'h…'
Token IDs (model-ready):
>>> phonemize("Hello world!", language="en-us").token_ids [ ... ]
Reusing a cached G2P instance:
>>> g2p = get_g2p(language="en-us") >>> phonemize("Hello world!", language="en-us", g2p=g2p).phonemes 'h…'
Full traceable result (tokens + warnings):
>>> span = [OverrideSpan(6, 10, {"lang": "de"})] >>> r = phonemize("Hello Welt!", language="en-us", overrides=span) >>> r.tokens[1].lang 'de'
Routing and target-model options
phonemize and phonemize_prepared accept the same routing and target-model controls:
overlap="split"applies partial language spans to exact source-aligned fragments. The default"snap"behavior remains compatible.language_routing={"mode": "auto", "languages": ["de", "en"]}enables optional pronunciation-language routing with a hard candidate allowlist. Any supported factory language, includingpt-pt, may be listed; the allowlist is never expanded implicitly.LanguageRoutingConfigis also accepted.g2p_resolver(language)supplies and caches a configured frontend per canonical language for one call. Without a custom resolver, foreign frontends use their own defaults and do not inherit the default frontend’s lexicons, version, or language-specific kwargs.target_model="1.0"makes the caller-supplied Kokoro vocabulary authoritative for routed phonemes and token IDs.
The document/default language is always explicit. Automatic routing does not detect the
document language. It inspects positive membership in the effective selected lexical
resources through G2PBase.lexicon_evidence(). Externally provisioned G2Lex evidence is
available for English US/GB and French. Provisioned Lexphon evidence is available for
German, Portuguese BR/PT, Russian, Thai, Vietnamese, Japanese, Korean, and Swedish when
NST is selected. Spanish, Italian, Czech, Hebrew, Arabic, Chinese, Kazakh, and Hindi
remain pronounceable but have no selected evidence provider in this release.
Generic lookup, proper-noun spelling, eSpeak, Goruut, pypinyin, Phonikud, g2pK,
pyopenjtalk, and rule fallback are not evidence. Lexicon collisions and ambiguity stay
in the default language. Candidate failures are isolated and warned. Explicit ph,
phonemes, lang, and language spans take precedence.
Language routing changes G2P frontend selection only. KokoroG2P does not select an
acoustic model or ONNX model. Routing decisions are available as structured
LanguageRoute objects in PhonemizeResult.language_routes, with selected lexicon
provenance. Do not pass arbitrary written text when number, date, unit, currency, or
abbreviation expansion is expected.
- kokorog2p.tokenize(text: str, language: str, keep_punct: bool = True) list[TokenSpan][source]
Convert text to a list of tokens with phonemes.
- Args:
text: Input text to convert. language: Language code (e.g., ‘en-us’, ‘en-gb’). keep_punct: Whether to include punctuation tokens.
- Returns:
List of TokenSpan objects with char offsets.
- Example:
>>> tokens = tokenize("Hello world!", language="en-us") >>> for t in tokens: ... print(f"{t.text} [{t.char_start}:{t.char_end}]") Hello [0:5] world [6:11] ! [11:12]
- kokorog2p.get_g2p(language: str, use_espeak_fallback: bool | None = None, use_goruut_fallback: bool = False, use_cli: bool = False, use_spacy: bool | None = None, backend: Literal['kokorog2p', 'espeak', 'goruut'] = 'kokorog2p', lexicons: str | Sequence[str] | None = None, version: str = '1.0', phoneme_quotes: str = 'curly', strict: bool = True, spacy_model: str | None = None, spacy_model_size: SpacyModelSize | None = None, **kwargs: Any) G2PBase[source]
Get a G2P instance for the specified language.
This factory function returns an appropriate G2P instance based on the
- Args:
language: Language code (e.g., ‘en-us’, ‘en-gb’, ‘zh’, ‘ja’, ‘fr’, etc.). use_espeak_fallback: Whether to use espeak for out-of-vocabulary words
when using the dictionary-based “kokorog2p” backend. Ignored when backend is set to “espeak” (espeak is the primary backend).
- use_goruut_fallback: Whether to use goruut for out-of-vocabulary words
when using the dictionary-based “kokorog2p” backend. Ignored when backend is set to “goruut” (goruut is the primary backend).
- use_spacy: Whether to use spaCy for tokenization and POS tagging. If
None, the highest-quality installed and loadable local model is attempted for languages with spaCy defaults and the native path is used when no model is available.Truerequires a loadable model;Falseforces the native path. No model is downloaded automatically.- spacy_model: Concrete spaCy model package name, or
"auto"for the highest installed loadable tier. Concrete names are strict.
- spacy_model_size: Exact model tier (
"trf","lg","md", or "sm") to select whenspacy_modelis unset.- use_cli: If True, force use of CLI espeak phonemizer instead of
library bindings. Only applies when backend=”espeak”.
- backend: Phonemization backend to use: “kokorog2p”, “espeak”, “goruut”.
The goruut backend requires pygoruut to be installed. Only applies to languages with dictionaries (English, French, German).
- lexicons: A named lexicon or ordered sequence of named lexicons for the
language. When omitted, the language default stack is used. In an explicit sequence, the first lexicon wins collisions. Use
available_lexicons(language)to inspect registered names.- version: Model version to use. Default: “1.0” (base model).
“1.0”: Base model
“1.1”: Chinese/English model
Different languages may have different behavior: - Chinese: “1.0” = IPA output, “1.1” = Zhuyin output
- phoneme_quotes: Quote character style in phoneme output. Options:
“curly”: Use curly quotes (”, “) - default, backward compatible
“ascii”: Use ASCII double quotes (“)
“none”: Remove quote characters from phoneme output
Only applies to English currently.
- strict: If True (default), raise exceptions when backend initialization
or phonemization fails. If False, log errors and return empty results for backward compatibility with older versions that silently failed. Recommended: True for production use to catch configuration issues.
**kwargs: Additional arguments passed to the G2P constructor.
- Returns:
A G2PBase instance for the specified language.
- Raises:
- ValueError: If the language is not supported and no fallback is available,
or if version is not “1.0” or “1.1”.
ImportError: If backend=”goruut” but pygoruut is not installed.
- Example:
>>> g2p = get_g2p("en-us") >>> tokens = g2p("Hello world!") >>> # Disable silver for better performance >>> # Fallback-only mode without a dictionary >>> # Chinese >>> g2p_zh = get_g2p("zh") >>> # Japanese >>> g2p_ja = get_g2p("ja") >>> # French (uses espeak fallback) >>> g2p_fr = get_g2p("fr") >>> # Using goruut backend >>> g2p_goruut = get_g2p("en-us", backend="goruut")
Pipeline integration adapters
These adapters accept SSMD- and phrasplit-shaped objects structurally, without making either package a runtime dependency:
- kokorog2p.coerce_override_spans(spans: Iterable[object]) list[OverrideSpan][source]
Copy and validate structurally compatible override spans.
The returned dataclasses own their attribute dictionaries, so adapting foreign spans never mutates caller-owned mappings.
- kokorog2p.overrides_from_ssmd(spans: Iterable[object], *, xsampa: Literal['reject', 'convert'] = 'reject', text_length: int | None = None) list[OverrideSpan][source]
Normalize SSMD-style annotation spans into kokorog2p overrides.
phand IPA-valuedipaattributes becomeph.langandlanguagebecomelang. X-SAMPA is rejected unless a converter is explicitly supplied by a future implementation; it is never silently interpreted as IPA.
- kokorog2p.overrides_for_segment(segment_start: int, segment_end: int, overrides: Iterable[object]) list[OverrideSpan][source]
Intersect document spans and rebase them to segment-local offsets.
- kokorog2p.phonemize_segments(clean_text: str, segments: Sequence[object], overrides: Sequence[object] = (), annotations: Sequence[TokenAnnotationLike] = (), *, phonemize: Callable[[...], PhonemizeResult], **kwargs: Any) list[PhonemizeResult][source]
Phonemize offset-preserving segments with document-level overrides.
Base Classes
G2PBase
- class kokorog2p.G2PBase(language: str = 'en-us', use_espeak_fallback: bool = True, use_goruut_fallback: bool = False, use_cli: bool = False, strict: bool = True)[source]
Bases:
ABCAbstract base class for grapheme-to-phoneme converters.
Subclasses must implement the __call__ method to convert text to phonemes.
- __init__(language: str = 'en-us', use_espeak_fallback: bool = True, use_goruut_fallback: bool = False, use_cli: bool = False, strict: bool = True) None[source]
Initialize the G2P converter.
- Args:
language: Language code (e.g., ‘en-us’, ‘en-gb’). use_espeak_fallback: Whether to use espeak for OOV words. use_goruut_fallback: Whether to use goruut for OOV words. use_cli: If True, use the CLI only for direct backend implementations. strict: If True, raise exceptions on errors. If False, log warnings
and return empty results (backward compatible mode).
- property is_british: bool
Check if this is British English.
- abstractmethod __call__(text: str) list[GToken][source]
Convert text to a list of tokens with phonemes.
- Args:
text: Input text to convert.
- Returns:
List of GToken objects with phonemes assigned.
- phonemize(text: str) str[source]
Convert text to a phoneme string.
This is a convenience method that calls __call__ and joins the results.
- Args:
text: Input text to convert.
- Returns:
Phoneme string with word boundaries.
- word_to_phonemes(word: str, tag: str | None = None) str | None[source]
Convert a single word to phonemes.
- Args:
word: The word to convert. tag: Optional POS tag for disambiguation.
- Returns:
Phoneme string or None if conversion failed.
- abstractmethod lookup(word: str, tag: str | None = None) str | None[source]
Look up a word in the dictionary.
- Args:
word: The word to look up. tag: Optional POS tag for disambiguation.
- Returns:
Phoneme string or None if not found.
- lexicon_evidence(word: str, tag: str | None = None) LexiconEvidence | None[source]
Return positive membership evidence from selected lexical resources.
GToken
- class kokorog2p.GToken(text: str, tag: str = '', whitespace: str = ' ', phonemes: str | None = None, start_ts: float | None = None, end_ts: float | None = None, rating: str | None = None, _: dict[str, ~typing.Any]=<factory>)[source]
Bases:
objectA token representing a word or text unit with optional phoneme information.
- Attributes:
text: The original text of the token. tag: Part-of-speech tag (e.g., ‘NN’, ‘VB’, ‘JJ’). whitespace: Trailing whitespace after this token. phonemes: The phonemic transcription of the token. start_ts: Start timestamp for audio alignment. end_ts: End timestamp for audio alignment. rating: Quality rating of the phoneme transcription. _: Extension dictionary for custom attributes.
- text
The original text of this token.
- phonemes
The IPA phoneme string for this token.
- tag
Part-of-speech tag (if available).
- whitespace
Whitespace following this token.
- text: str
- tag: str = ''
- whitespace: str = ' '
- property has_phonemes: bool
Check if this token has phonemes assigned.
- property is_punctuation: bool
Check if this token is punctuation.
- property is_word: bool
Check if this token is a word (not punctuation or whitespace).
Phoneme Utilities
Vocabulary
- kokorog2p.get_vocab(british: bool = False) frozenset[str][source]
Get the phoneme vocabulary for a dialect.
- Args:
british: Whether to get British or US vocabulary.
- Returns:
Frozen set of valid phonemes.
- kokorog2p.validate_phonemes(phonemes: str, british: bool = False) bool[source]
Check if all phonemes in a string are valid Kokoro phonemes.
- Args:
phonemes: Phoneme string to validate. british: Whether to validate against British or US vocabulary.
- Returns:
True if all phonemes are valid.
- kokorog2p.US_VOCAB
Build an immutable unordered collection of unique elements.
- kokorog2p.GB_VOCAB
Build an immutable unordered collection of unique elements.
- kokorog2p.VOWELS
Build an immutable unordered collection of unique elements.
- kokorog2p.CONSONANTS
Build an immutable unordered collection of unique elements.
Conversion
- kokorog2p.from_espeak(phonemes: str, british: bool = False) str[source]
Convert espeak IPA output to Kokoro phonemes.
- Args:
phonemes: The espeak phoneme string (with tie character ^ or ͡). british: Whether to use British English mappings.
- Returns:
Kokoro-compatible phoneme string.
- Example:
>>> from_espeak("mˈɜːt͡ʃənt͡ʃˌɪp", british=False) 'mˈɜɹʧəntʃˌɪp'
- kokorog2p.from_goruut(phonemes: str, british: bool = False) str[source]
Convert goruut/pygoruut IPA output to Kokoro phonemes.
Goruut outputs standard IPA without tie characters for diphthongs and affricates, which requires different handling than espeak.
- Args:
phonemes: The goruut phoneme string (standard IPA). british: Whether to use British English mappings.
- Returns:
Kokoro-compatible phoneme string.
- Example:
>>> from_goruut("həlˈoʊ wˈɜɹld", british=False) 'həlˈO wˈɜɹld' >>> from_goruut("sˈeɪ", british=False) 'sˈA'
Kokoro Vocabulary
Encoding/Decoding
- kokorog2p.encode(text: str, add_spaces: bool = True, model: str = '1.0') list[int][source]
Convert a phoneme string to token indices.
- Args:
text: Phoneme string to encode. add_spaces: Whether to include space tokens (default True). model: Model variant to encode for (default: “1.0”).
- Returns:
List of token indices.
- Example:
>>> encode("hˈɛlO") [50, 156, 86, 54, 31]
- kokorog2p.decode(indices: list[int], skip_special: bool = True, model: str = '1.0') str[source]
Convert token indices back to a phoneme string.
- Args:
indices: List of token indices. skip_special: Whether to skip padding/unknown tokens. model: Model variant to decode from (default: “1.0”).
- Returns:
Phoneme string.
- Example:
>>> decode([50, 156, 86, 54, 31]) 'hˈɛlO'
- kokorog2p.phonemes_to_ids(phonemes: str, model: str = '1.0') list[int][source]
Convert phoneme string to model input IDs.
This is the main function used to prepare text for the Kokoro model.
- Args:
phonemes: Phoneme string from G2P conversion. model: Model variant to encode for (default: “1.0”).
- Returns:
List of token IDs ready for model input.
- Example:
>>> phonemes_to_ids("hˈɛlO wˈɜɹld!") [50, 156, 86, 54, 31, 16, 65, 156, 87, 123, 54, 46, 5]
Validation
- kokorog2p.validate_for_kokoro(text: str, model: str = '1.0') tuple[bool, list[str]][source]
Validate that all characters in text are in Kokoro vocabulary.
- Args:
text: Phoneme string to validate. model: Model variant to validate against:
“1.0”: Base multilingual model (default)
“1.1”: Chinese-specific model with Zhuyin
- Returns:
Tuple of (is_valid, list_of_invalid_chars).
- Example:
>>> validate_for_kokoro("hˈɛlO") (True, []) >>> validate_for_kokoro("hˈɛlO§") (False, ['§']) >>> validate_for_kokoro("ㄋㄧ2ㄏㄠ3", model="1.1") (True, [])
- kokorog2p.filter_for_kokoro(text: str, replacement: str = '', model: str = '1.0') str[source]
Remove characters not in Kokoro vocabulary.
- Args:
text: Phoneme string to filter. replacement: String to replace invalid characters with. model: Model variant to filter for (same options as validate_for_kokoro).
- Returns:
Filtered phoneme string.
- Example:
>>> filter_for_kokoro("hˈɛlO§") 'hˈɛlO'
Configuration
- kokorog2p.get_kokoro_vocab(model: str = '1.0') dict[str, int]
Get the Kokoro vocabulary mapping (token -> index).
- Args:
- model: Model variant to load vocab for:
“1.0”: Base multilingual Kokoro model (default)
“1.1”: Chinese-specific Kokoro v1.1 model with Zhuyin
- Returns:
Dictionary mapping tokens to their indices.
- kokorog2p.get_kokoro_config() dict
Get the full Kokoro model configuration.
- Returns:
Dictionary containing the full model config.
- kokorog2p.N_TOKENS
int([x]) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.__int__(). For floating-point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ‘+’ or ‘-’ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer iteral. >>> int(‘0b100’, base=0) 4
- kokorog2p.PAD_IDX
int([x]) -> integer int(x, base=10) -> integer
Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.__int__(). For floating-point numbers, this truncates towards zero.
If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ‘+’ or ‘-’ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer iteral. >>> int(‘0b100’, base=0) 4
Punctuation
- class kokorog2p.Punctuation(marks: str | Pattern = ';:,.!?—…"()“”')[source]
Bases:
objectPreserve, remove, or normalize punctuation during phonemization.
This class provides methods to: 1. Normalize Unicode punctuation to Kokoro-compatible marks 2. remove configured marks 3. Preserve punctuation positions for later restoration
- Examples:
>>> punct = Punctuation()
# Normalize Unicode punctuation >>> punct.normalize(“Hello… world!”) ‘Hello… world!’
# Remove all punctuation >>> punct.remove(“Hello, world!”) ‘Hello world’
# Preserve and restore >>> text, marks = punct.preserve(“Hello, world!”) >>> text [‘Hello’, ‘world’] >>> # After phonemization… >>> punct.restore([‘həˈloʊ’, ‘wˈɜːld’], marks) [‘həˈloʊ, wˈɜːld!’]
- __init__(marks: str | Pattern = ';:,.!?—…"()“”')[source]
Initialize punctuation handler.
- Args:
- marks: Punctuation marks to consider. Either a string of
single-character marks or a compiled regex pattern.
- property marks: str
The punctuation marks as a string.
- normalize(text: str) str[source]
Normalize Unicode punctuation to Kokoro-compatible equivalents.
- Args:
text: Input text with various Unicode punctuation.
- Returns:
Text with normalized punctuation.
- Examples:
>>> punct = Punctuation() >>> punct.normalize("Hello… world!") 'Hello… world!' >>> punct.normalize('"Hello," she said.') '"Hello," she said.' >>> punct.normalize("Wait...what?!") 'Wait…what?!' >>> punct.normalize("don't worry") "don't worry" >>> punct.normalize("Wait - now") 'Wait — now'
- remove(text: str | list[str]) str | list[str][source]
Remove all punctuation marks, replacing with spaces.
- Args:
text: Input text or list of texts.
- Returns:
Text(s) with punctuation replaced by spaces.
- Examples:
>>> punct = Punctuation() >>> punct.remove("Hello, world!") 'Hello world' >>> punct.remove(["Hello!", "How are you?"]) ['Hello', 'How are you']
- preserve(text: str | list[str]) tuple[list[str], list[MarkIndex]][source]
Extract punctuation from text, preserving positions for restoration.
This splits the text into chunks without punctuation, while recording where each punctuation mark was located.
- Args:
text: Input text or list of texts.
- Returns:
Tuple of (text_chunks, mark_indices) where: - text_chunks: List of text segments without punctuation - mark_indices: List of MarkIndex objects for restoration
- Examples:
>>> punct = Punctuation() >>> text, marks = punct.preserve('Hello, world!') >>> text ['Hello', 'world'] >>> [(m.mark, m.position.value) for m in marks] [(', ', 'I'), ('!', 'E')]
- classmethod restore(text: str | list[str], marks: list[MarkIndex], word_sep: str = ' ', strip: bool = True) list[str][source]
Restore punctuation to phonemized text.
This is the reverse of preserve(). It takes phonemized text chunks and reinserts the punctuation marks at their original positions.
- Args:
text: Phonemized text chunks. marks: Mark indices from preserve(). word_sep: Word separator used in phonemized output. strip: Whether to strip trailing separators.
- Returns:
List of phonemized text with punctuation restored.
- Examples:
>>> punct = Punctuation() >>> text, marks = punct.preserve('Hello, world!') >>> punct.restore(['həˈloʊ', 'wˈɜːld'], marks) ['həˈloʊ, wˈɜːld!']
- kokorog2p.normalize_punctuation(text: str) str[source]
Normalize Unicode punctuation to Kokoro-compatible equivalents.
This is a convenience function that creates a Punctuation instance and calls normalize().
- Args:
text: Input text with various Unicode punctuation.
- Returns:
Text with normalized punctuation.
- Examples:
>>> normalize_punctuation("Hello… world!") 'Hello… world!'
- kokorog2p.filter_punctuation(text: str) str[source]
Keep only Kokoro-supported punctuation, remove everything else.
- Args:
text: Input text.
- Returns:
Text with only Kokoro-supported punctuation.
- Examples:
>>> filter_punctuation("Hello~world!") 'Hello world!'
- kokorog2p.is_kokoro_punctuation(char: str) bool[source]
Check if a character is a Kokoro-supported punctuation mark.
- Args:
char: Single character to check.
- Returns:
True if the character is in Kokoro’s punctuation vocabulary.
- kokorog2p.KOKORO_PUNCTUATION
Build an immutable unordered collection of unique elements.
Word Mismatch Detection
- class kokorog2p.MismatchMode(*values)[source]
Bases:
EnumHow to handle word count mismatches.
- IGNORE = 'ignore'
- WARN = 'warn'
- REMOVE = 'remove'
- class kokorog2p.MismatchInfo(line_num: int, expected: int, actual: int, input_text: str = '', output_text: str = '')[source]
Bases:
objectInformation about a word count mismatch.
- line_num: int
- expected: int
- actual: int
- input_text: str = ''
- output_text: str = ''
- class kokorog2p.MismatchStats(total_lines: int, mismatched_lines: int, mismatches: list[MismatchInfo])[source]
Bases:
objectStatistics about word count mismatches.
- total_lines: int
- mismatched_lines: int
- mismatches: list[MismatchInfo]
- property mismatch_rate: float
Percentage of lines with mismatches.
- kokorog2p.detect_mismatches(input_texts: list[str], output_texts: list[str], input_separator: str | Pattern[str] = re.compile('\\s+'), output_separator: str | Pattern[str] = re.compile('\\s+'), store_texts: bool = False) MismatchStats[source]
Detect word count mismatches between input and output.
- Args:
input_texts: Original input texts. output_texts: Phonemized output texts. input_separator: Word separator for input. output_separator: Word separator for output. store_texts: Whether to store input/output in MismatchInfo.
- Returns:
MismatchStats with details about any mismatches.
- Raises:
ValueError: If input and output have different lengths.
- Examples:
>>> inputs = ["hello world", "one two three"] >>> outputs = ["həˈloʊ wˈɜːld", "wˈʌn tuː θɹiː fɔːɹ"] # Extra word! >>> stats = detect_mismatches(inputs, outputs) >>> stats.mismatched_lines 1 >>> stats.mismatches[0].line_num 1
- kokorog2p.check_word_alignment(input_texts: list[str], output_texts: list[str], mode: MismatchMode | str = MismatchMode.WARN, input_separator: str | Pattern[str] = re.compile('\\s+'), output_separator: str | Pattern[str] = re.compile('\\s+'), logger: Logger | None = None) tuple[list[str], MismatchStats][source]
Check word alignment between input and output, optionally fixing issues.
This is a convenience function that combines detection and processing.
- Args:
input_texts: Original input texts. output_texts: Phonemized output texts. mode: How to handle mismatches (ignore, warn, remove). input_separator: Word separator for input texts. output_separator: Word separator for output texts. logger: Logger instance.
- Returns:
Tuple of (processed_outputs, statistics).
- Examples:
>>> inputs = ["hello world", "good morning"] >>> outputs = ["həˈloʊ wˈɜːld", "gʊd ˈmɔːnɪŋ ɛkstɹə"] >>> result, stats = check_word_alignment(inputs, outputs, mode="warn") >>> stats.mismatched_lines 1
- kokorog2p.count_words(text: str, separator: str | Pattern[str] = re.compile('\\s+')) int[source]
Count the number of words in text.
- Args:
text: Text to count words in. separator: Word separator (string or regex pattern).
- Returns:
Number of words.
- Examples:
>>> count_words("hello world") 2 >>> count_words("hello world") # Multiple spaces 2 >>> count_words("") 0