Whisper offers five different Whisper models, each with different accuracy and size. These models are called tiny, base, small, medium, and large-v2. The main difference between them is the number of parameters: the more parameters, the more accurately the model “understands” what it hears, and the fewer errors there are in transcription. Accordingly, smaller models will make more errors, such as confusing words.
When using larger models, it is important to note that decryption time increases, as do RAM and disk space requirements. This is a natural trade-off between accuracy and system resources.
In addition, the tiny, base, small, and medium models are available in a reduced version that only works with English. These versions are called tiny.en, base.en, small.en, and medium.en. Using language-specific models reduces transcription time and memory load, but they are only suitable for English audio.
It should be noted that recognition accuracy depends on the language. Not all languages provide the same high level of recognition when using Whisper. The WER (Word Error Ratio) metric is commonly used to evaluate quality. The lower the WER, the more accurately the model recognizes speech.
The large-v3 model demonstrates improved performance compared to previous versions and works with a larger number of languages. In tests on the Common Voice 15 and Fleurs datasets, large-v3 shows a 10–20% reduction in errors compared to the large-v2 model. This model is particularly useful when high accuracy and support for a variety of languages are required, although it also consumes more resources and time to process audio.
Therefore, when choosing a Whisper model, it is important to consider the balance between accuracy, speed, and system resources. Small models are suitable for fast and easy recognition, especially for English, while large models provide high accuracy for multiple languages and complex audio recordings.
