This document collection details SimpleFold, a protein structure prediction model from Apple Research. SimpleFold is presented as the first flow-matching based model that exclusively uses general-purpose transformer layers, deliberately avoiding complex, domain-specific modules like triangle attention common in state-of-the-art models such as AlphaFold. The model scales up to 3 billion parameters, making it the largest folding model developed to date. It was trained on a massive dataset of over 8.6 million structures, which includes both experimental data from the Protein Data Bank (PDB) and, critically, distilled structures previously generated by models like AlphaFold. The project claims competitive performance on standard benchmarks and strong capabilities in ensemble prediction. SimpleFold's primary contribution is to challenge the necessity of highly engineered architectures for protein folding, demonstrating that scaling simpler, more general models on large, distilled datasets is a viable path forward. The accompanying community discussion highlights that this "simplicity" is achieved by leveraging the knowledge of prior, more complex models, a process known as distillation.
SimpleFold: A Simplified Transformer-Based Approach to Protein Folding
Introducing SimpleFold: Apple has released SimpleFold, a new protein structure prediction model built on a standard transformer architecture and a generative flow-matching objective. It aims to simplify model design by removing the complex, specialized modules used in competing models.
Simplified Architecture: Unlike models such as AlphaFold that use highly engineered components (e.g., triangle attention), SimpleFold relies solely on general-purpose transformer layers. The community discussion notes this follows the "bitter lesson" in AI, where scaling general computation often outperforms complex, handcrafted features.
Training on Distilled Data: The model was trained on a massive 8.6M structure dataset. A key detail, highlighted in the discussion, is that this dataset is a hybrid of experimental PDB data and a large corpus of distilled structures previously predicted by other models (AlphaFold DB and AFESM). This means it learns from the outputs of more complex predecessors, which is crucial context for its simplified design.
Massive Scale: SimpleFold is available in various sizes, scaling from 100 million to 3 billion parameters. The 3B parameter model is described as the largest protein folding model ever developed.
Performance and Capabilities: The project reports competitive performance against state-of-the-art baselines. Its generative nature makes it particularly effective for ensemble prediction, allowing it to generate multiple diverse conformers for a single protein sequence.
Open Source and Accessible: The model is available as a Python package on GitHub, with pre-trained checkpoints available for download. It supports both PyTorch and MLX backends, making it optimized for efficient local inference on Apple Silicon hardware.
Licensing Details: The source code is released under a permissive MIT-style license. However, the pre-trained models are under a more restrictive "Apple Machine Learning Research Model License," which limits their use to non-commercial research purposes only.