Instructions to use WeiChow/RRSD with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use WeiChow/RRSD with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
RRSD: Learning from Relative Rewards via On-Policy Self-Distillation of Flow Models
Official Stable Diffusion 3.5 Medium (SD3.5-M) multi-reward LoRA checkpoints from Learning from Relative Rewards via On-Policy Self-Distillation of Flow Models.
Relative Reward Self-Distillation (RRSD) is a gradient-free on-policy post-training framework for diffusion and flow-matching generators. At selected intermediate denoising states, RRSD branches the trajectory with antithetic Gaussian perturbations, completes both rollouts through a shared deterministic ODE solver, cancels the shared rollout midpoint, and shifts the current clean prediction along a bounded relative-reward endpoint displacement to form a detached MSE target.
Available Checkpoints
Both checkpoints are rank-32 (lora_alpha=64) PEFT LoRA adapters trained for 100 outer rounds (115.2k primary trajectories) on stabilityai/stable-diffusion-3.5-medium under the shared multi-reward objective (PickScore + HPSv2 + GenEval + OCR) without classifier-free guidance (guidance_scale=1.0).
| Subfolder | Variant | States / Rollout | Terminal Evals | H100 GPU-h | LoRA Rank / Alpha | Format |
|---|---|---|---|---|---|---|
RRSD/ |
RRSD (Full) | 9 | 2.19M | 375 | 32 / 64 | PEFT safetensors |
RRSD-Lite/ |
RRSD-Lite | 4 | 1.04M | 185 | 32 / 64 | PEFT safetensors |
Benchmark Results (SD3.5-M Multi-Reward Post-Training)
Held-out results reported in Tables 1 and 2 of the paper (512Γ512, 40 deterministic Euler ODE steps, guidance_scale=1.0):
Preference, Compositional GenEval, and Text Rendering (Table 1)
| Model / Method | Traj. (k) | PickScore β | CLIPScore β | HPSv2 β | ImageReward β | GenEval Micro β | GenEval Macro β | GenEval Strict β | GenEval Soft β | OCR Acc. β |
|---|---|---|---|---|---|---|---|---|---|---|
| SD3.5-M (w/o CFG) | / | 20.51 | 0.237 | 0.204 | -0.58 | 0.2857 | 0.2972 | 0.1894 | 0.2314 | 0.1246 |
| + FlowGRPO | 115.2 | 22.30 | 0.267 | 0.290 | 0.81 | 0.5344 | 0.5498 | 0.4227 | 0.5763 | 0.5724 |
| + DiffusionNFT | 115.2 | 22.99 | 0.271 | 0.316 | 1.17 | 0.6623 | 0.6761 | 0.5416 | 0.6730 | 0.8584 |
| + Self-OPD | 115.2 | 23.45 | 0.270 | 0.333 | 1.21 | 0.6533 | 0.6671 | 0.5118 | 0.6415 | 0.7916 |
| RRSD-Lite | 115.2 | 23.55 | 0.276 | 0.342 | 1.37 | 0.7242 | 0.7357 | 0.6252 | 0.7569 | 0.8764 |
| RRSD | 115.2 | 23.74 | 0.281 | 0.346 | 1.42 | 0.7468 | 0.7592 | 0.6609 | 0.7703 | 0.8923 |
Compute Efficiency and Qwen-Image-Bench (Table 2)
| Method | Eval. (M) | GPU-h | Quality β | Aesthetics β | Alignment β | Fidelity β | Creativity β | Overall β |
|---|---|---|---|---|---|---|---|---|
| FlowGRPO | 0.12 | 54 | 42.99 | 40.06 | 31.29 | 37.19 | 19.51 | 35.50 |
| DiffusionNFT | 0.58 | 126 | 42.69 | 41.39 | 31.92 | 37.17 | 20.68 | 36.02 |
| Self-OPD | 2.19 | 398 | 44.26 | 40.03 | 31.97 | 37.29 | 21.95 | 36.33 |
| RRSD-Lite | 1.04 | 185 | 45.31 | 41.68 | 34.10 | 38.22 | 23.25 | 37.83 |
| RRSD | 2.19 | 375 | 45.68 | 42.43 | 34.61 | 38.68 | 23.75 | 38.32 |
Quickstart Usage (diffusers + peft)
import torch
from diffusers import StableDiffusion3Pipeline
from peft import PeftModel
base_model = "stabilityai/stable-diffusion-3.5-medium"
adapter_variant = "RRSD" # Choose "RRSD" or "RRSD-Lite"
pipe = StableDiffusion3Pipeline.from_pretrained(
base_model,
torch_dtype=torch.float16,
)
pipe.transformer = PeftModel.from_pretrained(
pipe.transformer,
"WeiChow/RRSD",
subfolder=adapter_variant,
is_trainable=False,
)
pipe.transformer.eval()
pipe = pipe.to("cuda")
prompt = 'A vintage brass compass resting on an old sea map with a banner reading "NORTHBOUND"'
generator = torch.Generator(device="cuda").manual_seed(42)
image = pipe(
prompt=prompt,
height=512,
width=512,
num_inference_steps=40,
guidance_scale=1.0,
generator=generator,
).images[0]
image.save("rrsd_sample.png")
License
The LoRA adapter weights and accompanying configuration files in this repository are released under the Apache-2.0 License. Usage of the base model stabilityai/stable-diffusion-3.5-medium remains subject to the upstream Stability AI Community License.
- Downloads last month
- -
Model tree for WeiChow/RRSD
Base model
stabilityai/stable-diffusion-3.5-medium