Eleven of them have no language code at all, so you cannot tell the model what it is listening to. It picks for you, and it picked wrong on all nineteen. Here is the full table, the method, and the one model that does better.
We shipped POST /v1/audio/transcriptions on 31 July and said, on the model page and in the docs,
that its African-language quality was not something we had measured. That was true and it was not
good enough, so we measured it. This is the result.
The short version: the model we serve is not usable for most African languages, and the reason is worse than a low score.
What we ran
FLEURS, the spoken counterpart of FLORES-200. Same sentences as the corpus behind our
language cost table, read aloud by native speakers, with reference transcripts. 19
African languages, 14 clips each, plus English and French as controls on the identical pipeline. 75
minutes of audio through whisper-large-v3-turbo, twice where possible: once letting the model
detect the language, once telling it.
We report character error rate rather than word error rate. Word error rate saturates. Once a model emits more wrong words than the reference contains, every failing language reads “1.0” and the number stops telling you which failures are worse.
Eleven languages have no code
Start here, because it is not a quality problem and no amount of data fixes it.
Whisper has a fixed set of language tokens. Igbo is not in it. Neither is Zulu, Xhosa, Wolof,
Chichewa, Luganda, Northern Sotho, Luo, Umbundu, Fula or Kamba. Sending language=ig does not
degrade gracefully, it returns a 422.
So a developer building for roughly 30 million Igbo speakers cannot tell the model it is hearing Igbo. The model then decides for itself, concludes it is hearing English, and returns fluent confident English-shaped text. No error. No warning. Nothing in the response says the thing you got back is unrelated to what was said.
We checked the same nineteen codes against Qwen3-ASR-1.7B, a different lab and a different
architecture. It accepts exactly the same eight and rejects exactly the same eleven. This is not a
Whisper problem. It is the shape of the field we can currently buy from.
The table
Character error rate, best of auto-detect and an explicit language hint. Lower is better.
| Language | CER | Language code | Auto-detected as |
|---|---|---|---|
| Afrikaans | 13% | yes | Afrikaans |
| Swahili | 20% | yes | Swahili |
| Lingala | 33% | yes | Swahili |
| Luganda | 33% | none | Swahili |
| Hausa | 43% | yes | Arabic |
| Kamba | 47% | none | Swahili |
| Yoruba | 48% | yes | Swahili |
| Umbundu | 49% | none | Swahili |
| Shona | 51% | yes | English |
| Chichewa | 52% | none | Swahili |
| Zulu | 53% | none | Swahili |
| Somali | 55% | yes | Arabic |
| Wolof | 63% | none | English |
| Luo | 65% | none | Swahili |
| Fula | 67% | none | English |
| Igbo | 70% | none | English |
| Northern Sotho | 75% | none | English |
| Xhosa | 92% | none | English |
| Amharic | 98% | yes | Arabic |
| French (control) | 8% | yes | French |
| English (control) | 26% | yes | English |
Three things in that table are worth sitting with.
Every single African language was auto-detected as something else. Hausa, Somali and Amharic as Arabic. Igbo, Xhosa, Wolof, Shona, Fula and Northern Sotho as English. Nine as Swahili. Nineteen languages, nineteen misidentifications.
Amharic is at 98%. Ge’ez script against a model that thinks it is hearing Arabic. That is not transcription with errors in it, that is a different document.
The best African result is Afrikaans, a Germanic language descended from Dutch, and the second best is Swahili, the most-resourced language on the continent. The ranking follows how much of each language was in the training data, which is the entire story in one column.
Where this measurement is weak
Our English control reads 26% CER, worse than published Whisper figures. Two reasons, both ours: one clip returned an empty transcription, and we apply no number normaliser, so a reference reading “twentieth century” scores as wrong against a perfectly correct “20th century”.
That inflates our European controls more than the African rows, because European errors here are orthographic and African ones are total. So read the gap, not the absolute figures. French at 8% is close to published values and gives you the floor of this pipeline.
14 clips per language establishes an order of magnitude. It does not rank two close models. The two-fold gaps below are real; a four-point difference is not.
One model does better, and it is already available
We ran the same clips through every speech model our supplier serves. Character error rate, no language hint, so each model gets the same information:
| Model | Swahili | Hausa | Yoruba | Igbo | Zulu | Amharic |
|---|---|---|---|---|---|---|
whisper-large-v3-turbo | 11% | 80% | 52% | 77% | 42% | 125% |
qwen3-asr-1.7b | 15% | 47% | 58% | 48% | 38% | 120% |
| Voxtral Mini 3B | 48% | 54% | 53% | 70% | 38% | 121% |
| Nemotron 3.5 ASR | 74% | 78% | 83% | 82% | 66% | 100% |
whisper-large-v3 | 75% | 72% | 397% | 142% | 201% | 105% |
qwen3-asr-1.7b is now in the catalogue. It is better than Whisper on Hausa, Igbo and Zulu, and
worse on Swahili. It costs $0.00045 per audio minute against Whisper turbo’s $0.00020, which for a
three-minute voice note is the difference between $0.002 and $0.0009.
We do not route between them for you. Picking a model per detected language would change your bill and your accuracy based on a guess you never saw, and language detection is precisely what these models are worst at. Both are published with their numbers. You choose.
Two more things in that table. whisper-large-v3, the larger model, is dramatically worse than the
turbo variant, not better: 397% CER on Yoruba means it generated four times more text than was
spoken, looping until it hit a limit. And nothing works on Amharic. Every model is at or above 100%.
What we are doing about it
Adding a second model is not a fix, it is a slightly better floor. The real answer is that Meta released Omnilingual ASR under Apache 2.0 in late 2025: 1,600+ languages, character error rate under 10% on 78% of them, and it covers all eleven languages nothing we can buy has a code for. Its published figures for the rows above are 2.8% Swahili, 3.5% Hausa, 9.1% Igbo, 4.9% Xhosa.
No serverless provider hosts it. Serving it means running it ourselves, which is a real piece of work and not a promise we are making today. We are pricing it.
Until then, the numbers above are on /models beside every speech model, the languages with no code
are flagged there, and both are on GET /v1/models as speech_accuracy_cer and no_language_token
so you can read them from a client instead of a page.
If you are transcribing an African language today, check your language in that table first. For eleven of them the honest answer is that we cannot yet do this well, and you should know that before you build on it rather than after.