Jyutping Romanization

Among the most common tasks in handling Cantonese corpus data are those that involve the processing of Jyutping romanization. A common need is to convert Cantonese characters to Jyutping romanization. Another functionality of interest is the ability to convert Jyutping into other romanization schemes still used today. Whether you have data in Jyutping from a corpus reader or you have independently ingested Jyutping as Python strings, PyCantonese provides tools for these use cases.

Characters-to-Jyutping Conversion

The function characters_to_jyutping() takes a string of Cantonese characters and returns its word-segmented version with Jyutping romanization:

>>> import pycantonese
>>> pycantonese.characters_to_jyutping('香港人講廣東話')  # Hongkongers speak Cantonese
[('香港人', 'hoeng1gong2jan4'), ('講', 'gong2'), ('廣東話', 'gwong2dung1waa2')]

The characters-to-Jyutping conversion model is based on two data sources: (i) the HKCanCor corpus data included in the PyCantonese library, and (ii) the rime-cantonese data (the 2021.05.16 release, CC BY 4.0 license). Any unseen character, Cantonese or otherwise, is represented by None in the output.

To further process the Jyutping strings, please see Parsing Jyutping Strings.

A Cantonese character may have multiple pronunciations, most commonly due to pinjam (變音, “changed tone”). Whether the function characters_to_jyutping() can intelligently output the correct, contextually dependent pronunciation depends on whether the underlying data contains the relevant tokens. Example:

>>> import pycantonese
>>> # The correct pronunciation of 蛋 is with tone 2 (high-rising) as a standalone word.
>>> pycantonese.characters_to_jyutping('蛋')  # egg
[('蛋', 'daan2')]
>>>
>>> # The correct pronunciation of 蛋 is with tone 6 (low-level) in 蛋糕.
>>> pycantonese.characters_to_jyutping('蛋糕')  # cake
[('蛋糕', 'daan6gou1')]

Because characters_to_jyutping() performs word segmentation under the hood (via segment()), it is possible to customize word segmentation by passing in a Segmenter instance to the segmenter keyword argument of characters_to_jyutping().

>>> import pycantonese
>>> from pycantonese.word_segmentation import Segmenter
>>> # Create a `Segmenter` class instance.
>>> # See its documentation for what customization it allows.
>>> # As an example, the `disallow` parameter can take an iterable of strings
>>> # that represent words that you don't want to treat as words.
>>> # Here, let's pretend that you don't want 蛋糕 to be segmented as a single word.
>>> my_segmenter = Segmenter(disallow={"蛋糕"})
>>> pycantonese.characters_to_jyutping("蛋糕", segmenter=my_segmenter)
[('蛋', 'daan2'), ('糕', 'gou1')]

Parsing Jyutping Strings

Converting Jyutping to other romanization schemes necessitates the ability to parse Jyutping for the various phonological components (onset, nucleus, coda, and tone). To this end, PyCantonese exposes the function parse_jyutping() which parses a string of Jyutping romanization and returns a list of Jyutping objects; the string may contain results for multiple Chinese characters.:

>>> import pycantonese
>>> pycantonese.parse_jyutping('hou2')  # 好 good
[Jyutping(onset='h', nucleus='o', coda='u', tone='2')]
>>> pycantonese.parse_jyutping('gwong2dung1waa2')  # 廣東話 Cantonese
[Jyutping(onset='gw', nucleus='o', coda='ng', tone='2'),
 Jyutping(onset='d', nucleus='u', coda='ng', tone='1'),
 Jyutping(onset='w', nucleus='aa', coda='', tone='2')]

Syllabic nasals are treated as nuclei:

>>> import pycantonese
>>> pycantonese.parse_jyutping('m4goi1')  # 唔該 thank you / please
[Jyutping(onset='', nucleus='m', coda='', tone='4'),
 Jyutping(onset='g', nucleus='o', coda='i', tone='1')]

The function parse_jyutping() is able to detect invalid Jyutping romanization:

>>> import pycantonese
>>> pycantonese.parse_jyutping('hou7')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.9/dist-packages/pycantonese/jyutping.py", line 197, in parse_jyutping
    raise ValueError('tone error -- ' + repr(jp))
ValueError: tone error -- 'hou7'

The Jyutping class makes it easy to access the onset, nucleus, coda, and tone using the attribute syntax. It is also straightforward to retrieve the string representation and final (= nucleus + coda; 韻母):

>>> from pycantonese.jyutping import Jyutping
>>> jp = Jyutping(onset="j", nucleus="yu", coda="t", tone="6")
>>> jp.onset
'j'
>>> jp.nucleus
'yu'
>>> jp.coda
't'
>>> jp.tone
'6'
>>> str(jp)
'jyut6'
>>> jp.final
'yut'

Jyutping-to-Yale Conversion

The Yale romanization is still a commonly used system, particularly in numerous dictionaries and Cantonese language teaching resources. PyCantonese provides the jyutping_to_yale() function which reads a valid Jyutping string and returns the Yale equivalent:

>>> import pycantonese
>>> pycantonese.jyutping_to_yale('m4goi1')  # 唔該 thank you / please
['m̀h', 'gōi']
>>> pycantonese.jyutping_to_yale('gwong2dung1waa2')  # 廣東話 Cantonese
['gwóng', 'dūng', 'wá']

jyutping_to_yale() has the keyword argument as_list. When set to be False, it turns the returned value into a string.

>>> import pycantonese
>>> pycantonese.jyutping_to_yale('gwong2dung1waa2', as_list=False)  # 廣東話 Cantonese
'gwóngdūngwá'

While getting a string instead of a list might seem trivial enough that as_list would be necessary, its usefulness arises when there is potential confusion. In Yale romanization, a consonant letter or the low-tone marker “h” can be ambiguous as an onset of a syllable or as part of the previous syllable. When such ambiguity is detected, as_list=False automatically adds the quote character ' as a separator to disambiguate:

>>> import pycantonese
>>> pycantonese.jyutping_to_yale('hei3hau6', as_list=False)  # 氣候 climate
"hei'hauh"
>>> # 'heihauh' would be ambiguous between hei3hau6 and hei6au6.

Jyutping-to-TIPA Conversion

PyCantonese also offers the jyutping_to_tipa() function for the LaTeX TIPA users:

>>> import pycantonese
>>> pycantonese.jyutping_to_tipa('m4goi1')  # 唔該 thank you / please
['\\s{m}21', 'kOY55']
>>> pycantonese.jyutping_to_tipa('gwong2dung1waa2')  # 廣東話 Cantonese
['k\\super w ON25', 'tUN55', 'wa25']

Currently, tones are output as Chao tone letters (= the numbers from 1 to 5) directly suffixed to the individual syllable string. (This may change in a future release if this behavior proves to be inconvenient.)