Finite Difference Flow Optimization

RL Post-Training for Flow-Based Image Generators

Finite Difference Flow Optimization Finite Difference Flow Optimization RL Post-Training for Flow-Based Image Generators Epoch 0 Epoch 50 Results from before and after our human preference RL post-training. No CFG is used in either image. David McAllister Miika Aittala Tero Karras Janne Hellsten Angjoo Kanazawa Timo Aila Samuli Laine Paper arXiv Code In this project, we set out to find a simple, grounded RL post-training method for diffusion image generators. We made a few observations about the structure of diffusion flows that lead to Finite Difference Flow Optimization (FDFO), a new online

saved by

• Rajan Agarwal

related reading

• Flow Matching Policy Gradientsflowreinforce.github.io
• Diffusion models from scratchchenyang.co
• What are Diffusion Models? | Lil'Loglilianweng.github.io
• Elucidating the Design Space of Diffusion-Based Generative Models | PDFarxiv.org
• Flow Matching for Generative Modelingalphaxiv.org
• Learning the integral of a diffusion model – Sander Dielemansander.ai
• What are Diffusion Models?lilianweng.github.io
• Diffusion Meets Flow Matchingdiffusionflow.github.io
• [2305.18290] Direct Preference Optimization: Your Language Model is Secretly a Reward Modelarxiv.org
• SFT, RL, and On-Policy Distillation Through a Distributional Lens | whnrehiew.github.io
• [2006.11239] Denoising Diffusion Probabilistic Modelsarxiv.org
• ⭐️ Diffusion Modelsandrewkchan.dev