This document introduces SimpleFold, the first flow-matching based protein folding model that utilizes solely general-purpose transformer layers, omitting complex domain-specific modules like triangle attention. Developed by Apple, SimpleFold is trained on an exceptionally large corpus of over 8.6 million distilled protein structures combined with experimental PDB data, scaling up to a 3-billion parameter model—the largest folding model reported. It achieves competitive performance on standard folding benchmarks and demonstrates strong performance in ensemble prediction due to its generative objective. The repository provides comprehensive instructions for installation and inference, supporting both PyTorch and Apple's MLX framework (with a demonstration on M2 Max hardware), and offers different model sizes from 100M to 3B parameters, challenging the established reliance on highly engineered architectures in protein structure prediction.
SimpleFold: Folding Proteins is Simpler than You Think (Project Summary and Discussion)
Project Overview (Introduction): SimpleFold is introduced as a flow-matching based protein folding model relying solely on general-purpose transformer layers, notably excluding complex modules like triangle attention.
Model Scale and Training Data: The largest version, SimpleFold-3B, comprises 3 billion parameters and was trained on over 8.6 million distilled protein structures (from AFDB SwissProt and AFESM) along with experimental PDB data.
Performance and Objective: The model achieves competitive performance against state-of-the-art baselines and excels in ensemble prediction due to its generative flow-matching training objective.
Installation and Inference: The package supports inference via both PyTorch and Apple's MLX framework (recommended for Apple hardware). Users can choose from six model sizes ranging from 100M to 3B parameters.
Evaluation: Predicted structures for standard benchmarks (CAMEO22, CASP14, Apo, CoDNaS) are provided, with instructions for evaluation using tools like OpenStructure and TM-score.
Training and Data Preparation (Data preparation/Training): Detailed instructions are given for setting up training, including processing mmCIF files (requires a local Redis server instance) and utilizing configurations compatible with standard (default) and FSDP (Fully Sharded Data Parallel) strategies.
Licensing Constraints (License_Model): The released SimpleFold models are provided under an Apple Machine Learning Research Model License, which strictly limits usage to Research Purposes (non-commercial scientific research and academic development); commercial exploitation or use in commercial products is explicitly forbidden.
Discussion Context (Hacker News):
Distillation Critique: Critics noted the training data relies heavily on structures predicted by existing complex models (like AlphaFold), suggesting the model inherits their inductive biases rather than truly demonstrating simplicity from scratch.
Architectural Value: Proponents argue the breakthrough lies in proving that a simple transformer architecture can achieve competitive results, highlighting the efficacy of architectural simplification (a trend seen historically in ML development).
Versus Classical Methods: SimpleFold/AlphaFold use fast statistical methods based on known folds, contrasting with older, expensive physics-based solvers like Folding@Home.
Apple's Motivation: The development is seen as part of Apple's general ML research, potentially focusing on optimizing large models for local, low-latency inference on Apple silicon (e.g., M-series chips).