Why Cantonese transcription drops words

Explainer · 2026-09-13

If you have tried an English-first meeting recorder on a Hong Kong meeting, you know the output: a paragraph that is roughly the right length, with the English words intact, the Mandarin approximated, and the Cantonese missing or rewritten into something nobody said. This post is about why that happens and what we do differently. It has no benchmark numbers in it — those come next month, with the clips — because the failure modes are worth understanding on their own.

1. Spoken Cantonese is not written Chinese

A Cantonese speaker says 佢哋唔係咁講 and, when writing formally, writes 他們不是這樣說. Different characters, different words, same meaning. Speech models trained mostly on written Chinese text (subtitles, transcripts normalised by editors) learn to output the written form — so the transcript says 他們, the speaker said 佢哋, and a reader checking the transcript against the recording reasonably concludes it is wrong. Some models go further and refuse the colloquial characters entirely, dropping the words they cannot map.

What Audiee does: the transcript keeps Cantonese as spoken — 佢哋, 唔係, 咁 — because the transcript's job is to be checkable against the audio. The summary is written in standard written Chinese, because that is what you paste into an email. Two registers, two artefacts, on purpose.

2. The language changes every few words

「呢個 proposal 我哋 next week 再 confirm」 is one sentence with three switches. Most recognisers decide the language once per utterance, or once per meeting from a setting. An English-first model hears "proposal" and "next week" and "confirm" clearly and treats the rest as noise between them; a Mandarin model does the reverse and renders "proposal" as a phonetic approximation. Either way, half the sentence goes.

What Audiee does: Mixed is the default language, and Mixed, Mandarin and Cantonese are all routed to a CJK-first model (BytePlus's Seed ASR) that was trained on code-switched speech, rather than to the streaming English-first engine we use for other languages. Every meeting takes this path regardless of the setting, because meetings are where the switching happens. The routing is a few lines of code; we will show them to anyone who asks.

3. Particles are treated as noise

啦, 喎, 囉, 咩, 啫 carry the meaning of a Cantonese sentence — whether it is a question, a concession, a reminder. Acoustically they are short and unstressed, exactly what a model tuned for word-error-rate on English learns to drop. A transcript without them is grammatical and wrong.

What Audiee does: nothing clever — the CJK-first model keeps them, and we do not post-process them away. We mention it because several tools run a "cleanup" pass that removes filler words, and in Cantonese that pass removes grammar.

4. Loanwords and names

Hong Kong English inside Cantonese is its own vocabulary: "book 咗 room", "send 個 email", "個 deadline 好 tight". Recognisers either anglicise ("booked the room") or sinicise (簿左房). Names are worse: a colleague called Wing, a client called 陳生, a project called Phoenix, and the model has never heard any of them together.

What Audiee does: your personal vocabulary is sent to the recogniser as inline hotwords at the start of each session — names of people and projects from Knowledge, terms you have corrected, cased and with aliases. Since September every polished summary adds its proper nouns to that list, so the vocabulary grows with your meetings rather than with your patience. Rename a speaker once and the recogniser hears the new name next time.

5. Two people, one channel

This is a trade-off we made, and you should know it. To get the CJK model's accuracy we send it a single mixed channel, so the model does not know which words came from your microphone and which from the call. Speaker turns therefore come from Zoom's active-speaker tags (read from Zoom's window on your Mac, with the Accessibility permission), not from the audio. On Zoom this is accurate; on Meet and Teams in a browser it is experimental and you will see "Speaker 1" until you rename.

What we cannot fix

• Crosstalk. Two people speaking at once is lost in every system we have tested, ours included.
• Bluetooth headsets that open their microphone late: the first half-second of each turn is gone before it reaches any model. Wired or built-in microphones do not have this problem.
• Very noisy rooms, and speakerphones across a conference table.
• Sentences in a fourth language. Mixed covers English, Mandarin and Cantonese; a Japanese phrase in the middle will be approximated.

How to check us

Record a two-minute Cantonese meeting on the free plan, then read the transcript against the audio. Look for the particles, look for the names, look for whether 佢哋 survived. If something breaks, email the clip to hello@audiee.ai — real failing clips are how the benchmark set is being built, and how the vocabulary defaults get better. Setup is in the Mandarin and Cantonese guide.