Here is the abstract and summary for the provided transcript:
Abstract: GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models
This paper introduces GLM-4.5, an open-source Mixture-of-Experts (MoE) large language model with 355 billion total and 32 billion activated parameters, alongside a compact version, GLM-4.5-Air (106 billion parameters). The models feature a novel hybrid reasoning method, enabling both deliberative thinking and direct response modes. Through a multi-stage training process on 23T tokens, followed by comprehensive post-training involving expert model iteration and reinforcement learning, GLM-4.5 achieves state-of-the-art performance across Agentic, Reasoning, and Coding (ARC) tasks. It scores 70.1% on TAU-Bench, 91.0% on AIME 24, and 64.2% on SWE-bench Verified, ranking 3rd overall and 2nd on agentic benchmarks among evaluated models, despite using significantly fewer parameters than many competitors. The release aims to advance research in reasoning and agentic AI systems.
Summarizing the Research Paper: GLM-4.5
0:00 Introduction of GLM-4.5 Series: GLM-4.5 (355B total, 32B activated parameters) and GLM-4.5-Air (106B parameters) are introduced as open-source Mixture-of-Experts (MoE) large language models. They feature a hybrid reasoning method supporting both "thinking" for complex tasks and "direct response" modes.
0:45 Model Architecture (Sec 2.1): The models adopt an MoE architecture with loss-free balance routing and sigmoid gates. They prioritize a deeper model (more layers) over wider models for better reasoning. Innovations include 2.5 times more attention heads (96 heads for 5120 hidden dim) and QK-Norm for attention stability. An MoE layer is also used for Multi-Token Prediction (MTP) to support speculative decoding.
1:15 Pre-Training Data (Sec 2.2): The corpus includes 23T tokens from diverse sources like webpages (English & Chinese, quality-scored, SemDedup for de-duplication), social media, books, papers, and code repositories. High-quality data is up-sampled, and Fill-In-the-Middle objective is applied to code.
2:20 Mid-Training Stages (Sec 2.3): Post-pre-training, several medium-size, domain-specific datasets are used to boost performance:
Repo-level Code Training: Concatenated code files, model-filtered GitHub issues, pull requests, and commits are used to learn cross-file dependencies and software engineering capabilities. Sequence length is extended to 32K.
Synthetic Reasoning Data Training: Synthetic content for math, science, and coding competitions, with synthesized reasoning processes.
Long-context & Agent Training: Sequence length extended from 32K to 128K, up-sampling long documents and incorporating large-scale synthetic agent trajectories.
3:00 Post-Training: Expert Model Iteration (Sec 3): The process is divided into two stages:
Expert Training: Construction of specialized expert models for Reasoning, Agent, and General chat.
Unified Training: Self-distillation techniques integrate multiple experts into a comprehensive model for hybrid reasoning.
3:15 Supervised Fine-Tuning (SFT) (Sec 3.1):
Cold Start SFT: Provides foundational abilities with Chain-of-Thought (CoT) responses.
Overall SFT: Millions of samples from expert models are used to distill capabilities into a generalist model, balancing full reasoning and instant responses.
Key Strategies: Novel XML-like function call templates are used to reduce character escaping issues in code segments (Figure 4). A comprehensive multi-stage filtering pipeline is applied for rejection sampling, and prompt selection/response-level scaling improve performance on challenging tasks. Automatic agentic SFT data construction is used for tool-use scenarios.
5:50 Reasoning RL (Sec 3.2): Enhances capabilities in logical deduction, math, code, and science with precise reward signals.
Difficulty-based Curriculum Learning: A two-stage curriculum adapts to model proficiency, ensuring useful gradient signals (Figure 5).
Single-Stage RL at 64K Output Length: Experiments show a single RL stage at max target length (64K) is more effective than progressively increasing lengths, as multi-stage can lead to "unlearning" long-context capabilities (Figure 6).
Dynamic Sampling Temperature: Adjusts temperature during RL to balance exploration and accuracy.
Code and Science RL: Token-weighted mean loss improves code RL efficiency, and high-quality, expert-verified data is crucial for science RL (Figure 7).
8:40 Agentic RL (Sec 3.3): Focuses on web-search and code-generation agents where actions can be automatically checked.
Data Collection & Synthesis: Pipeline for demanding web-search tasks and extensive GitHub pull requests/issues for software engineering.
Training & Distillation: Uses group-wise policy optimization, outcome supervision with a process action format penalty. Iterative self-distillation enhances SFT models by substituting cold-start data with RL-trained model responses.
Scaling Test-time Compute: Significant performance gains observed with increasing interaction turns with the environment for agent tasks (Figure 8).
10:15 General RL (Sec 3.4): Aims to improve overall performance, remediate issues, and strengthen key capabilities using a multi-source feedback system (rule-based, human, model-based).
Holistic RL: Uses a balanced dataset and reward signals from human and AI feedback.
Instruction Following RL: Creates a fine-grained taxonomy of constraint types, using deterministic verification rules and reward models (Figure 9).
Function Calling RL: Includes step-wise rule-based RL (strict reward function) and end-to-end multi-turn RL for autonomous planning and decision-making.
Pathology RL: Targets specific pathological behaviors (language mixing, repetition) with curated datasets and efficient penalties.
11:50 RL Infrastructure (Sec 3.5): Built upon Slime, an open-source framework, designed for flexibility, efficiency, and extensibility. Supports hybrid training (colocated synchronous and disaggregated asynchronous modes), accelerated rollout with mixed-precision inference (BF16 for training, FP8 for inference), and agent-oriented design for long-horizon rollouts (Docker-based runtime, asynchronous training loop).
13:50 Evaluation of Base Models (Sec 4.1): GLM-4.5-Base shows stable performance across English, Code, Math, and Chinese benchmarks, validating its generalist ability.
15:00 Agentic Abilities (Sec 4.2.1): GLM-4.5 performs well on TAU-bench (Retail & Airline), BFCL V3 (best overall among baselines), and BrowseComp (close to o4-mini, outperforms Claude Opus 4) (Table 3).
16:00 Reasoning Abilities (Sec 4.2.2): Demonstrates strong performance on MMLU-Pro, AIME 24, MATH 500, SciCode, GPQA, HLE, and LiveCodeBench, outperforming OpenAI o3 on AIME 24 and SciCode, and competitive with top models (Table 4).
16:50 Coding Abilities (Sec 4.2.3): Excels on SWE-bench Verified (outperforms GPT-4.1 and Gemini-2.5-Pro) and Terminal-Bench (outperforms Claude Sonnet 4), positioning as a strong competitor for coding tasks (Table 5).
17:30 Evaluation of General Abilities (Sec 4.2.4): Competitive performance on MMLU, SimpleQA, IFEval, SysBench, and MultiChallenge benchmarks, often outperforming or matching models with significantly more parameters (Table 6).
18:15 Evaluation of Safety (Sec 4.2.5): Achieves a strong overall safety score (89.87) on SafetyBench, competitive with top-tier models like Kimi-K2 and GPT-4.1, with high performance in Ethics, Mental Health, and Physical Health (Table 7).
19:15 Evaluations for Hands-on Experience (Sec 4.3): Human evaluation framework established to assess real-world application capabilities and qualitative aspects.
19:40 General Chat (Sec 4.3.1): GLM-4.5 achieved the highest overall scores in human evaluations for English, Chinese, and other languages, excelling in areas like Mathematics, Text Generation, and Logical Reasoning (Tables 8, 9, 10).
20:55 Coding Agent (Sec 4.3.2): On CC-Bench, GLM-4.5 demonstrated strong performance against Claude Sonnet 4, Kimi K2, and Qwen3-Coder (winning 80.8% vs Qwen3-Coder), with the highest tool calling success rate (90.6%) (Figure 12, 13).
22:00 Logical Reasoning (Sec 4.3.3): Performed competitively on a new, challenging set of logical reasoning problems, scoring on par with leading models like Gemini 2.5 Pro and DeepSeek-R1 (Table 11).
22:40 Evaluation of Translation (Sec 4.4): Significantly outperforms specialized translation models on 100 challenging, real-world cases that require nuanced cultural and contextual understanding (e.g., internet slang, domain nicknames, emojis) (Table 12).
23:45 Conclusion (Sec 5): GLM-4.5 series, utilizing MoE architecture, excels in reasoning, coding, and agentic tasks, achieving a 3rd global rank among models, and is released open-source to advance AI research.
Summarizing the Hacker News Discussion:
0:15 Appreciation for Depth (starchild3001):
Users appreciate the paper's technical depth ("how" not just "what") which is a welcome change from typical model announcements.
Key Takeaway: The detailed post-training methodology, especially the creation and distillation of specialized "expert models" (for reasoning, agents, chat) into a unified generalist model, is seen as a fascinating and structured approach to avoid the "jack of all trades, master of none" problem.
A significant finding highlighted is that a single-stage RL process at the full 64K context length outperforms progressive multi-stage approaches.
The pragmatic engineering decision to use an XML-like template for function calls (Figure 4) to avoid "JSON escaping hell" is lauded as "small but brilliant" and making a "huge difference in practice."
Key Takeaway: Practical insights from the paper's RL experiments and engineering choices are highly valued by those working in the LLM space.
A user who has used GLM-4.5 as a coding model (in Octofriend) finds it "really, really good," sometimes confusing it with Claude 4.
Key Takeaway: Subjective experience indicates GLM-4.5 is highly competitive for coding, particularly in its "honesty" (fixing bugs rather than manipulating tests) and ability to catch issues missed by Claude models.
1:45 Direct Problem-Solving Success (UncleOxidant):
A user reports GLM-4.5 quickly resolving a tricky compiler bug that Gemini CLI had failed to identify all day.
Discussion ensues regarding how to use GLM-4.5 (API vs. local Air version).
Users share experiences with API costs ($0.20/1M tokens, leading to $30 in hours for one user) and the impact of caching on price comparisons.
Privacy concerns are raised about data storage (Synthetic.new storing data for 14 days vs. Fireworks' 0 logging).
Key Takeaway: While highly capable, practical deployment of GLM-4.5 involves considerations of cost and data privacy, especially when using third-party APIs.
3:10 Overall Positive Reception:
Multiple users praise GLM-4.5 as "a great model for software development" and "probably the best of the freely available ones."
It's seen as the "first open model that doesn’t require significant caveats when comparing to frontier proprietary models," highlighting its parameter efficiency and potential innovations.
The Apache license is welcomed, supporting open-source progress.
Key Takeaway: GLM-4.5 is widely recognized as a significant leap forward for open-source LLMs, especially in coding, and is seen as a strong competitor to proprietary models.
4:10 Comparative Strengths and Areas for Further Verification:
One user places GLM-4.5's visual reasoning "somewhere between a qwen 2.5 VL and the frontier models (o3 / opus 4)."
Requests are made for independent verification of results, such as evaluation on Aider’s LLM Leaderboard.
Key Takeaway: While strong, further independent benchmarking and specific comparisons (e.g., visual reasoning) are desired.
4:30 Broader Impact:
One comment expresses "Huge respect to the open-source culture in China," stating they are "really leading the world in democratizing AI."
Key Takeaway: The release is seen as a testament to China's growing leadership in open-source AI development.