This video clarifies the two fundamental training stages for large language models (LLMs) like Gemma: pre-training and instruction tuning, to assist users in selecting the optimal model for their projects. Pre-training focuses on next-token prediction using vast datasets, enabling the model to internalize world knowledge. Instruction tuning subsequently converts this predictor into a functional chatbot, imbuing it with the ability to follow specific commands, adopt a helpful persona, and format outputs in diverse ways. The presenter recommends instruction-tuned models for general-purpose, ready-to-use applications and suggests pre-trained models for highly specialized fine-tuning tasks that require unique output structures or a bias-free starting point. A crucial technical detail regarding the distinct special tokens used by each model type is also covered.
Summarizing the Gemma Training Stages: Pre-trained vs. Instruction-tuned Models
0:00:11 Introduction to LLM Training Stages: Leonar from Google DeepMind introduces the two key phases of Large Language Model (LLM) training for Gemma: pre-training and instruction tuning, to help users select the appropriate model.
The model is trained using cross-entropy loss, learning next-token prediction from massive amounts of text and images.
Key takeaway: This phase allows the model to compress world information and implicitly learn knowledge representations, becoming a "next token predictor."
Occurs after pre-training, transforming the next token predictor into an actual chatbot.
Uses specific data and formatting to teach the model to answer user queries, rather than just predicting the next most probable token.
Key takeaway: This shapes the model's behavior and personality, making it helpful, safe, capable of reasoning, and able to follow a wide set of instructions (e.g., generating JSON, writing in different languages, controlling paragraph count).
Recommended for ready-to-go applications that need to understand and follow instructions, answer questions effectively, and are generally more capable in areas like math or coding out-of-the-box.
Key takeaway: For fine-tuning tasks that align with general instruction following (e.g., specialized Q&A bots) and can utilize Gemma's existing conversational format, the instruction-tuned model provides a performance head start.
0:02:21 When to use the Pre-trained Model for Fine-tuning:
Ideal if you need a completely different output format that doesn't resemble a conversation, or if your task is very niche and far removed from typical interactions.
Key takeaway: The pre-trained model offers a "cleaner slate" for greater flexibility, allowing users to shape it for unique needs without having to "unlearn" instruction-tuned behaviors (e.g., translating dolphin language, generating highly specific scientific data).
For pre-trained models: Use BOS (Beginning Of Sequence) to start input and EOS (End Of Sequence) to indicate the model is done generating.
For instruction-tuned models: Start with BOS, but use specific "turn markers" to differentiate user input and model response. Generation stops when an "end of turn" token is produced (not EOS).
Key takeaway: Be aware that the library you use to run Gemma might handle these special tokens automatically.
0:04:13 Conclusion: Gemma provides powerful pre-trained and helpful instruction-tuned open-source LLMs, offering developers versatile tools for exploration, experimentation, and building innovative products.