Starting with version 2.0.0, Whishper supports GPU usage, which significantly speeds up the process of decrypting audio files. Using a video card is especially useful when working with large amounts of data or long recordings.
Currently, this feature has only been tested on GNU/Linux amd64 systems with NVIDIA RTX graphics cards. To use the GPU, you need to install the cuBLAS and cuDNN libraries for CUDA 11, as specified in the faster-whisper library requirements. Support for video cards from other manufacturers is not guaranteed and depends on the capabilities of the faster-whisper library itself. If you are using a different architecture or operating system, it is recommended that you test the GPU and report any issues you encounter to help improve support.
To use the GPU, you must have an NVIDIA graphics card with CUDA support. You can check compatibility on the official CUDA website. After that, you need to configure your operating system correctly. There are ready-made configuration instructions for popular Linux distributions: Fedora, Arch Linux, Ubuntu.
You also need to install the NVIDIA Container Toolkit so that Docker can use the GPU. Installation and configuration instructions are available on the official NVIDIA website. After completing all the steps, the system will be ready to work with the GPU.
You can enable GPU support in Whishper during installation using the get-whishper.sh script—it will prompt you to enable the GPU, automatically download the necessary files, and configure the environment variables. If you want to activate the GPU later, you can do so by replacing the standard docker-compose.yml file with docker-compose.gpu.yml. Before running, make sure that all requirements are met. Enabling the GPU also activates acceleration for the LibreTranslate service.
Updating Whishper from versions 2.x to 3.x
The update process is quite simple: replace the docker-compose.yml file with the latest version of the GPU compose file and check the environment variable settings in the .env file. The COMPOSE_PROFILES variable is no longer required. After that, simply restart the containers using the docker-compose up -d command. The first time you run it, the download may be slower, as the new GPU image contains all CUDA and Torch dependencies and is significantly larger (about 7 GB versus 450 MB for the CPU version).
Updating Whishper from versions 1.x to 2.x
To update from version 1.x, you need to replace docker-compose.yml with the current compose file, set the COMPOSE_PROFILES variable in the .env file to gpu, and restart the containers. This will enable Whisper to work with the GPU.
Thus, GPU support allows you to speed up audio processing, reduce decryption time, and increase efficiency when working with large amounts of data, but it requires proper system configuration and installation of all necessary libraries and tools.
