This video explores Mistral's Voxil language model, emphasizing its capabilities in speech understanding, transcription, and innovative function calling. The presenter demonstrates the Voxil Mini (3 billion parameter) variant running locally, highlighting its substantial 32K context length for processing long audio inputs (up to 40 minutes) and its native multilingual support. Practical applications of the function calling feature are showcased through voice-activated games, including a simple text adventure and a Blackjack simulation, where spoken commands directly control game logic. While commending the model's accuracy and potential for advanced voice control (e.g., smart home integration), the video also addresses the significant VRAM consumption (20GB for the 3B model) and the current complexities of local setup, especially on newer GPU architectures like Nvidia's Blackwell series.
Exploring Mistral's Voxil: Voice Control and Function Calling in Action
0:00 Introduction to Voxil: Mistral has released Voxil, a new language model capable of speech understanding, transcription, and unique function calling capabilities, demonstrated initially with a voice-activated Blackjack game.
0:19 Model Comparison: Voxil is presented as a capable model, similar to Quen 2.5 Omni and Gemma 3N in its ability to transcribe and understand speech.
0:41 Key Feature: Function Calling: A significant highlight of Voxil is its ability to perform "function calling" directly from voice commands, opening up possibilities for interactive applications and control.
1:14 Model Variants: Voxil is available in two versions: a 24 billion parameter variant (Voxil) and a smaller 3 billion parameter variant (Voxil Mini), with the latter being used for the demonstration due to VRAM considerations.
1:37 Extended Context Length: The model boasts a 32K context length, allowing it to handle up to 30 minutes of audio for transcription or 40 minutes for understanding, making it suitable for longer audio inputs like TV show episodes.
2:18 Natively Multilingual: Voxil is designed to be natively multilingual, supporting various languages beyond English.
2:43 Future Development: Planned improvements for Voxil include non-speech audio recognition, speaker segmentation, audio markup (age and emotion detection), and word-level timestamps.
3:17 Local Deployment Challenges: Running Voxil locally, particularly with VLLM (vLLM), can be challenging, especially with newer GPU architectures like Nvidia's Blackwell (5000 series).
3:47 High VRAM Usage: The Voxil Mini (3B) model uses approximately 20GB of VRAM when run with VLLM, higher than the stated 9GB, largely due to VLLM's allocation for the full context length.
4:57 Transcription Demonstration: A 3-minute 24-second audio clip of Obama's speech is transcribed, showcasing the model's accuracy and speed.
6:15 Voice-Controlled Text Adventure: A simple command-line text adventure game demonstrates Voxil's function calling by allowing players to control game actions (e.g., "Look around," "Walk north") using voice commands.
8:04 Interactive Blackjack Game: A web-based Blackjack game highlights the power of voice-activated function calling, enabling players to bet, hit, and stand using spoken commands, illustrating potential for smart home control.
10:43 Conclusion and Accessibility: Voxil is presented as a very cool and capable model, but its current downside for hobbyists is the complexity of VLLM setup and significant VRAM requirements, particularly with newer GPU hardware. The larger 24B variant offers more power but demands even greater computational resources.