This video introduces Mistral's new Voxil language model, a powerful tool capable of speech understanding, transcription, and advanced voice-activated function calling. Focusing on the 3-billion parameter "Voxil Mini" variant, the presenter highlights its impressive 32K context length, allowing it to process and transcribe extended audio segments (up to 40 minutes). The model also boasts native multilingual support and plans for future enhancements like non-speech audio recognition. The core demonstration showcases Voxil's accuracy in transcribing a long speech and creatively illustrates its function-calling capabilities through interactive voice-controlled games, including a text adventure and blackjack, underscoring its potential for practical applications such as smart home automation. While highly capable, the video notes challenges for local deployment by hobbyists, particularly concerning its significant VRAM consumption (around 20GB for the Mini model) and setup complexities with certain GPU architectures.
Exploring Mistral's Voxil: Voice Understanding, Transcription, and Function Calling
0:00 Introduction to Voxil: Mistral has released Voxil, a language model capable of both understanding and transcribing speech, with a key highlight being its function calling capabilities demonstrated through a voice-activated blackjack game.
0:18 Model Variants: Voxil is available in a 24-billion parameter version and a more accessible 3-billion parameter "Mini" variant. The smaller Mini model is used for the video's demonstrations.
1:38 Extensive Context Length: A significant feature is Voxil's 32K context length, enabling it to handle up to 30 minutes of transcription or 40 minutes of speech understanding, hypothetically capable of transcribing entire audio files of TV show episodes.
2:16 Multilingual and Function Calling: Voxil is natively multilingual and excels at "function calling straight from voice," which opens up possibilities for controlling physical tasks, such as smart home devices.
2:43 Future Development: Planned improvements include non-speech audio recognition, speaker segmentation, audio markup (age/emotion), and word-level timestamps.
3:17 Local Deployment Notes: Running Voxil locally, especially with VLLM on newer hardware like the Nvidia Blackwell architecture (5000 series), can be challenging and resource-intensive.
4:08 High VRAM Usage: Although the Hugging Face model card suggests 9GB of VRAM, the Voxil Mini model consumes approximately 20GB of VRAM during VLLM usage, attributed to allocating for its large context length.
4:57 Transcription Demonstration: The model successfully transcribes a 3-minute 24-second audio clip of an Obama speech quickly and with high clarity, indicating a low word error rate.
6:16 Interactive Voice Control Demos: To showcase function calling, the video presents two simple voice-activated games: a text adventure (using commands like "Look around," "walk north," "move west") and a blackjack game (using commands like "Bet 100," "Hit," "Stand").
10:53 Setup Hurdles for Hobbyists: The presenter notes that setting up the LLM for personal use can be cumbersome, particularly for users with newer GPU series.
11:19 Model Accessibility: The video concludes by reiterating the availability of both the 3B (Mini) and 24B parameter Voxil models, with the larger model requiring substantial computational capability.