This news article from ETH Zurich announces the upcoming release in late summer 2025 of a large, open-source, and highly multilingual Large Language Model (LLM), a collaborative effort by EPFL, ETH Zurich, and the Swiss National Supercomputing Centre (CSCS). Developed and trained on the "Alps" supercomputer using 100% climate-neutral power, this initiative marks a significant advancement for open and trustworthy AI. The LLM aims to foster innovation across science, society, and industry by providing a transparent alternative to commercial systems, emphasizing its capability in over 1000 languages and adherence to responsible data practices.
Ein Sprachmodell im Dienste der Gesellschaft: A New Open-Source LLM from Swiss Universities
0:00 Introduction of a Groundbreaking LLM: ETH Zurich and EPFL, in collaboration with the Swiss National Supercomputing Centre (CSCS), are set to release a large, open-source, and multilingual Large Language Model (LLM) in late summer 2025. This model was developed and trained on the public infrastructure of the "Alps" supercomputer, marking a significant step for open AI.
0:45 Key Features and Goals: This LLM will be fully open, with publicly available source code, weights, and transparent, reproducible training data. This approach aims to enable broad utilization and foster innovation across science, society, and industry, serving as a trustworthy alternative to commercial, non-transparent systems.
1:30 Transparency and Collaboration: The initiative was previewed at the International Open-Source LLM Builders Summit in Geneva, hosted by EPFL and ETH Zurich AI centers. The project emphasizes transparent development processes to promote trust and ensure compliance with regulations like the EU AI Act.
2:40 Advanced Multilingual Capabilities: A core feature is its proficiency in over 1000 languages. The base model was trained on a vast dataset comprising over 1500 languages (60% English, 40% other languages), alongside code and mathematical data, ensuring global applicability.
3:50 Scalable and Robust Design: The model will be released in two sizes: 8 billion and 70 billion parameters, with the larger version expected to be among the most powerful fully open models globally. It is trained with over 15 trillion high-quality tokens for robust language understanding and versatile applications.
4:50 Commitment to Responsible Data Handling: The development adheres to Swiss data protection and copyright laws, and the transparency obligations of the EU AI Act. Research associated with the project indicates that respecting web crawling "opt-outs" (excluding certain web content) results in minimal performance impact for most common tasks.
6:00 Powered by "Alps" Supercomputer: The LLM is trained on the "Alps" supercomputer at CSCS in Lugano, featuring over 10,000 NVIDIA Grace Hopper Superchips. Its architecture allows for efficient training with 100% climate-neutral power, leveraging long-standing collaborations with NVIDIA and HPE/Cray to build independent AI infrastructure.
7:20 Open Access and Future Development: The LLM will be released under an Apache-2.0 license, accompanied by comprehensive documentation detailing its architecture, training methods, and usage guidelines. This promotes transparent reuse and empowers organizations to build their own applications on top of the model.
8:30 Part of a Broader Swiss AI Initiative: This project is a central component of the "Swiss AI Initiative," launched in December 2023 by EPFL and ETH Zurich, supported by over ten Swiss scientific institutions. With over 800 researchers involved and access to significant GPU hours, it represents the world's largest open-science and open-source project for AI base models, supported by the ETH Board and aligned with European AI networks like ELLIS.