On 12 October 2026, a preprint posted to arXiv described a reinforcement learning technique that can fine-tune large video diffusion models using an order of magnitude less GPU time than two prior methods, the authors report.

The method, called DiffusionART, matched the reward levels of DiffusionNFT and Flow-GRPO-Fast on HunyuanVideo-13B in 12.02 and 11.72 times fewer GPU hours, respectively, while finishing with 28.8% and 53.0% higher mean training rewards, according to the paper. For labs and studios that repeatedly run RL fine-tuning on such models, each 12× reduction in GPU time could slash the per-run cost by a similar factor, though the preprint does not disclose the absolute hardware bill or the GPU instance types used.

Online reinforcement learning is an attractive way to adapt pretrained diffusion models using black-box rewards—for example, from human preference scores—but current algorithms trade off efficiency against stability. Forward-process approaches such as DiffusionNFT are efficient yet require careful tuning of an exponential-moving-average (EMA) mechanism to avoid collapse, while reverse-process methods like Flow-GRPO are stable but slow because of weak credit assignment, the authors write.

The preprint argues that the NFT objective contains a hidden “forward–reverse consistency force” that destabilises training unless partially cancelled by a jointly trained negative branch. EMA, far from providing a broader data distribution, actually works by restoring the model toward a lagged reference policy, the paper contends.

DiffusionART, short for Diffusion Reinforcement with Aggregation-First Reward Tilting, removes the consistency force by construction. It aggregates reward signals and endpoint latent vectors for each prompt before regression, generating a single reward-improving direction for all visited reverse-process states. The algorithm then performs two disjoint minibatch updates: the first follows the guidance, while the second adaptively pulls the model back toward the frozen rollout policy using a trust-region mask, doing away with EMA altogether. No differentiable reward model is needed, and the method supports pairwise preferences as well as scalar rewards.

Computational Savings

The head-to-head comparison on HunyuanVideo-13B used VideoReward with equal weights on visual and motion quality. The GPU-hour tallies, which included evaluation and waiting time, allowed DiffusionART to reach the baselines’ late-stage mean rewards faster, and then to climb higher before the run ended. The paper does not report the absolute GPU hours for this experiment, but the project page provides a separate image-to-video benchmark: on Wan2.1-I2V-14B, DiffusionART reached a motion-quality benchmark in 45.8 GPU hours versus 220.3 for Flow-GRPO-Fast, a 4.8× reduction.

For text-to-image, the algorithm needed 10.0×, 5.2×, and 9.6× fewer training steps than Flow-GRPO to match its step‑500 rewards on the GenEval, OCR, and PickScore benchmarks using SD3.5-M, according to the same project page. A pairwise video variant, DiffusionART-Pairwise, with only two samples per prompt required 23.4× fewer GPU hours over 400 updates than a K=8 configuration while achieving a comparable visual-quality evaluation score (12.24 vs 12.09 on the authors’ scale).

The preprint is not peer-reviewed, and no third party has replicated the results. The hardware configuration behind the GPU-hour counts is not specified, so the absolute cost savings remain unquantified. Additionally, the comparisons are against specific versions of DiffusionNFT and Flow-GRPO-Fast, and the rewards are derived from reward models whose own biases are not discussed.

Method and Release

Alongside the paper, the team released a GitHub repository that packages the method into a YAML-configurable training framework for text-to-image, text-to-video, and image-to-video tasks. It includes fine-tuned weights for SD3.5-M and HunyuanVideo-13B on Hugging Face, as well as training recipes for models including FLUX.1-dev and Wan2.1-I2V-14B. Six baseline algorithms—among them Flow-GRPO, DanceGRPO, and DiffusionNFT—are also provided for comparison, the repository states.

The preprint is available as arXiv:2610.12572.