Fine-tuning Flow Matching Generative Models with Intermediate Feedback
Published in arXiv preprint (Under Review), 2025, 2025
Recommended citation: Jiajun Fan, Chaoran Cheng, Shuaike Shen, Xiangxin Zhou, Ge Liu. "Fine-tuning Flow Matching Generative Models with Intermediate Feedback." arXiv:2510.18072, 2025. https://arxiv.org/abs/2510.18072
We present AC-Flow, a robust actor-critic framework for fine-tuning flow matching generative models with intermediate feedback. Key innovations include reward shaping for stable intermediate value learning, a dual-stability mechanism combining advantage clipping with critic warm-up, and a scalable generalized critic weighting scheme with Wasserstein regularization. Achieves SOTA text-to-image alignment on Stable Diffusion 3 without compromising diversity or stability.