TL;DR — Inverse-folding models are trained to copy curated sequence–structure data, which is not the same objective as designing a sequence that actually folds and stays stable. ProteinZero closes that loop with online RL: the model samples its own designs, scores them with ESMFold and a self-derived fast ddG predictor, and improves continuously — with an embedding-level diversity regularizer that stops the usual collapse onto a few safe sequences. All evaluation is computational.

The problem

Supervised inverse folding optimizes sequence recovery against curated datasets. The design goal is different: a sequence that refolds to the target backbone and is predicted to be stable. Preference-based fine-tuning (DPO) narrows that gap, but it learns from a fixed, pre-collected preference set — and in our runs it pays for its gains with diversity.

Offline fine-tuning

Learns from a fixed preference set. The model cannot explore beyond it, and under repeated rounds it concentrates on a narrow set of high-reward sequences — measured diversity falls below the base model.

ProteinZero (online RL)

The model generates, scores and learns from its own designs each round. A diversity term defined on sequence embeddings keeps the policy spread out, so designability and predicted stability improve without collapsing.

Method

1

Sample

The policy proposes sequences for a target backbone — its own rollouts, not a dataset.

2

Score

ESMFold gives structural feedback; a self-derived fast ddG model gives a cheap predicted-stability signal.

3

Stay diverse

A diversity regularizer on embedding space penalizes collapse onto near-identical designs.

4

Update

GRPO (or RAFT) with a KL term to the reference model, balancing multiple rewards.

The fast ddG predictor is what makes this affordable: physics-based stability evaluation would dominate training cost, so it is replaced by a learned proxy inside the loop. A complete run fits on one 8-GPU node in about three days, including reward computation and data generation.

Scope: fast ddG is a training proxy. Its experimental validation is moderate and based on single-point mutations, while it is applied here to full-sequence redesign, so it is not a calibrated predictor of absolute folding stability.

Results on CATH-4.3

90.1%
success rate, 0–150 residues
91.2%
success rate, 150–300 residues
36–48%
fewer design failures vs. prior models
3 days
one 8-GPU node per run
2026-10-12T17:32:06.184338 image/svg+xml Matplotlib v3.10.8, https://matplotlib.org/ ESM-IF ProteinMPNN InstructPLM (base) DPO Multi-Round DPO ProteinZero (RAFT) ProteinZero (GRPO) 75 80 85 90 95 success rate (%) 80.7 82.0 84.5 86.4 86.9 89.3 90.1 82.8 84.7 86.4 87.7 88.0 89.4 91.2 solid = 0-150 residues faded = 150-300 residues
Figure 1. Success rate on held-out CATH-4.3 backbones — a design counts only if it satisfies both criteria (scRMSD < 2 Å and FoldX ddG < 0). Both ProteinZero variants pass every supervised and preference-based baseline; GRPO is the stronger of the two.

The reward uses ESMFold and fast ddG, but the gains also appear under independent evaluators (AlphaFold3 for structure, FoldX for stability) that were never part of training — evidence that the improvements are not simply fitted to the reward models. All of this is in-silico; no wet-lab validation is performed.

Diversity is the part that usually breaks

2026-10-12T17:32:06.204237 image/svg+xml Matplotlib v3.10.8, https://matplotlib.org/ InstructPLM (base) DPO Multi-Round DPO ProteinZero (GRPO) 0.26 0.28 0.30 0.32 sequence diversity 0.281 0.274 0.266 0.306 base model
Figure 2. Sequence diversity among generated designs (0–150 residues). Both DPO variants end up below the base model — the classic collapse onto a few high-reward sequences — while ProteinZero raises diversity and success rate together.

The regularizer acts on the decoder's sequence embeddings rather than on raw token overlap, so it spreads designs apart in representation space while the reward keeps pulling them toward foldable, stable candidates. Where the diversity signal enters turns out to matter far more than whether it is present: pushing it into the reward instead costs more than ten points of success rate.

2026-10-12T17:35:09.292066 image/svg+xml Matplotlib v3.10.8, https://matplotlib.org/ Hamming distance as reward diversity as reward embedding-level regularizer (ours) 70 75 80 85 90 95 success rate (%) 74.63 78.65 90.13 Hamming distance as reward diversity as reward embedding-level regularizer (ours) 0.26 0.28 0.30 0.32 sequence diversity 0.276 0.284 0.306
Figure 3. Three ways to ask for diversity (0–150 residues). Rewarding Hamming distance or adding diversity to the reward drags the policy away from foldable, stable designs; keeping it as a separate embedding-level term does not.

Reported honestly: dropping the diversity term altogether leaves success essentially unchanged (90.23% vs. 90.13%) while diversity falls from 0.306 to 0.268. The term buys sequence diversity at roughly no cost in success rate — it is not what produces the headline number.

Code

ProteinZero is open source at github.com/ziwenwang28/ProteinZero. It is usable as a component in a larger pipeline — for example as a design tool called by an agent — and collaborations are welcome.

BibTeX

@article{wang2026proteinzero,
  title   = {ProteinZero: Self-Improving Protein Generation via Online
             Reinforcement Learning},
  author  = {Ziwen Wang and Jiajun Fan and Ruihan Guo and Thao Nguyen
             and Heng Ji and Ge Liu},
  journal = {Transactions on Machine Learning Research},
  year    = {2026},
  url     = {https://arxiv.org/abs/2506.07459}
}