This paper introduces GLM-4.5, a powerful open-source Mixture-of-Experts (MoE) large language model developed by Zhipu AI and Tsinghua University. The model, with 355 billion total and 32 billion active parameters, is engineered to excel in agentic, reasoning, and coding (ARC) tasks. Its architecture and a sophisticated multi-stage training process, involving 23 trillion tokens and subsequent reinforcement learning with an innovative "expert model iteration" technique, enable it to achieve top-tier performance. GLM-4.5 ranks 3rd overall among evaluated models and 2nd on agentic benchmarks, demonstrating high parameter efficiency compared to its competitors. The paper also presents a smaller version, GLM-4.5-Air (106B parameters), with both models being released to the open-source community to foster advancements in agentic AI and complex reasoning systems.
Introducing GLM-4.5: An Open-Source MoE Model Excelling in Agentic, Reasoning, and Coding Tasks
1. A New Frontier Model: GLM-4.5 is introduced as a highly capable, open-source Mixture-of-Experts (MoE) model designed to unify agentic, reasoning, and coding (ARC) abilities. It achieves a 3rd place overall ranking against proprietary competitors while being significantly more parameter-efficient.
2.1 Efficient MoE Architecture: The model features a 355B total parameter count with 32B active parameters per token. The architecture is deliberately designed to be "deeper" (more layers) rather than "wider" to enhance reasoning capabilities. It also incorporates a novel XML-like template for function calls to reduce complexity and improve performance on agentic tasks.
2.2 Multi-Stage Training: The model undergoes extensive training, beginning with a 15T token pre-training phase on a diverse corpus of web data, code, and scientific papers. This is followed by several "mid-training" stages on domain-specific data (e.g., repo-level code, synthetic reasoning) where the context length is extended up to 128K.
3.0 Innovative Post-Training: A key innovation is the "Expert Model Iteration" methodology. First, specialized expert models are trained for Reasoning, Agentic tasks, and General Chat. Then, these specialized capabilities