verl-project/verl-omni

Call for contributions: RL & distillation algorithms for diffusion & omni post-training We'd love your help bringing new post-training algorithms into verl-omni. The shortlist below is all published work with runnable reference code, and each fits the loss registry and recipe structure already in the repo — so a contribution is a focused PR, not a research project. Treat it as suggestions, not a closed menu — other algorithms are welcome too, and a short RFC is a good way to start. If one of these catches your eye, please comment here first so we can coordinate. 1. DGPO — Direct Group Preference Optimization (ICLR 2026) Paper: arXiv 2510.08425 · Code: Luo-Yihong/DGPO Extends DPO from pairs to whole groups and drops the policy gradient entirely, which unlocks deterministic ODE samplers — the paper reports ~20× faster training than existing diffusion RL. 2. TDM-R1 — RL for few-step models with non-differentiable rewards (ICML 2026) Paper: arXiv 2603.07700 · Code: Luo-Yihong

Dashboard RFC: take MiniMax-H3 from working LoRA recipes to production-scale training. Land incremental PRs; do not treat this as one change. Foundation already on main: T2VA #347/#368/#383, FL2VA #396/#401/#448, Ref2VA #458/#464/#485, V1 sync T2VA/FL2VA #501, FLOPs/MFU #548. 1. Pipelines Official H3 tasks are t2va / fl2va / ref2va. There is no H3 ti2va; image-conditioned generation is FL2VA (first / last / first+last keyframes). Task DiffusionNFT FlowGRPO V1 sync GPU NPU T2VA ✅ ✅ ✅ ✅ FlowGRPO only FL2VA ✅ ✅ ✅ ✅ ❌ Ref2VA ✅ ✅ ✅ ✅ ❌ Gaps: Ref2VA V1 sync (and later separate_async once checkpoint-engine sync exists). Reference-aware rewards for Ref2VA (CLAP/ImageBind do not measure reference fidelity). FL2VA / Ref2VA NPU (optional; T2VA FlowGRPO NPU already exists). Default recipes already cover first-frame FL2VA; last / first+last are data-path supported via frame_mode. 2. VeOmni actor backend (default candidate) VeOmni is already wired in

1. Summary This RFC proposes adding DiffusionNFT (direct-preference forward-process RL) support for the BOOGU-Image model family, covering both text-to-image (Boogu-Image-0.1-Base) and instruction-guided image editing (Boogu-Image-0.1-Edit). Currently, BooguImagePipeline only supports policy-gradient FlowGRPO (algorithm="flow_grpo"). While FlowGRPO performs well, it requires collecting reverse-SDE trajectories and log-probabilities across all denoising steps, incurring high rollout host memory and network transfer overhead. DiffusionNFT trains from final clean latents ($x_0$) using a forward-process objective, enabling: Significant rollout memory reduction: Omits trajectory latent/logprob caching, slashing rollout-to-actor tensor payload by ~10–20×. Native deterministic ODE generation: Rollout uses native Euler/ODE inference without stochastic SDE perturbation, avoiding likelihood drift. High sample efficiency for image editing: Provides stable preference optimization for both text

Feature request Build the DMD2 FSDP engine and offline trainer integration for the Qwen-Image MVP in #519, using the loss/config work in #520 and the concrete adapter/acceptance work in #543. Following cr-gao's review, replace the general role-runtime/controller/executor design with a concrete DMD2 integration. Draft PR #546 implements this reusable runtime slice; the concrete Qwen adapter and recipe are reviewed separately under #543. Motivation DMD2 needs differentiable sampling, detached fake-stage samples and independent student/fake optimization, not another Ray lifecycle or arbitrary role/transport framework. Adapt the existing worker/engine's single-active-optimizer assumptions rather than treating DMD2 as PPO replay or an OPD request. Only DMD2 distribution-only is in scope. Original DMD, GAN, causal Wan and Self-Forcing are not prerequisites. Proposed structure main_diffusion / existing task and resource-pool setup -> DistributionMatchingRayTrainer(BaseRayDiffusionTrainer)

Feature request Implement the DMD2 loss and standalone DMD configuration for the Qwen-Image MVP in #519. This replaces this issue's former generic-controller/immutable-plan scope, following the review of #545. This is a CPU-verifiable foundation for the concrete engine/trainer and Qwen integration in #535/#543. It does not introduce a production trainer shell, a four-recipe registry, or claim runnable training by itself. Existing #545 code still follows the previous design until it is updated; prior passing tests do not certify this revision. Motivation Separate algorithm identity, shared mathematics, and execution: Original DMD (dmd, deferred) combines distribution matching with paired teacher-trajectory regression. DMD2 (dmd2, MVP) removes that paired regression and trains the student against frozen real-score and trainable fake-score predictions with two-timescale optimization. Fake-score denoising MSE is not original DMD's paired generator-regression term. A class named DMDLoss re