PixelDense-XXL T2I

PixelDense: Dense Prediction as Representation Alignment for Pixel Diffusion (NeurIPS 2026)

📄 Paper | 🌐 Project Page | 💻 Code

PixelDense uses frozen dense-prediction models as representation-alignment targets for pixel-space diffusion. DINOv2 and SAM2 supervise a semantic projection stream, Depth Anything v2 and Metric3D v2 supervise a geometric projection stream, and a weight-space orthogonality penalty keeps the two streams in disjoint subspaces of the denoiser feature. The teachers and alignment projections are used only during training, so this checkpoint is a plain 1.1B pixel-space text-to-image denoiser with the same inference cost as PixelGen.

Model details

Architecture	JiT-T2I pixel-space denoiser (PixelGen-XXL)
Parameters	1.1B
Resolution	512 × 512
Text encoder	Qwen3-1.7B (not included; downloaded automatically)
Sampler	Adam LM solver, 25 steps, classifier-free guidance 4.0
Base model	PixelGen-XXL T2I
Fine-tuning data	BLIP3o-60k

See the paper for the full training recipe.

Files

File	Size	Contents
PixelDense_XXL_T2I.ckpt	4.5 GB	EMA denoiser weights, fp32

The checkpoint holds weights only: {"state_dict": {"ema_denoiser.<param>": tensor}}. It has no optimizer state or training metadata and loads with torch.load(path, weights_only=True). Use it with the PixelDense code: the denoiser has one learned embedding (visual_type_embedding) that upstream PixelGen does not have.

Usage

git clone https://github.com/Hansxsourse/PixelDense
cd PixelDense
pip install -r requirements.txt

hf download xsourse/PixelDense PixelDense_XXL_T2I.ckpt --local-dir ./ckpts
python main.py predict -c ./configs_t2i/pixeldense_inference_res512.yaml

Images are written to ./workdirs/exp_pixeldense_inference/images/predict/. To use your own prompts, pass a text file with one prompt per line:

python main.py predict -c ./configs_t2i/pixeldense_inference_res512.yaml \
    --data.pred_dataset.init_args.conditions ./prompts.txt

Inference needs only the denoiser and the Qwen3 text encoder; the dense teachers are not loaded.

Results

512 × 512, from Table 1 of the paper. Higher is better for every metric.

Model	GenEval Overall	DPG-Bench	HPS v2.1
PixelGen (released)	0.79	–	0.281
PixelGen + fine-tune (DINOv2 REPA only)	0.7927	78.7	0.280
PixelDense	0.8093	78.9	0.282

"PixelGen + fine-tune" is the matched baseline: the same fine-tuning run with only the original DINOv2 alignment.

Limitations

This is a research checkpoint. It generates at 512 × 512 only, and its outputs can reflect biases in the base model's training data and in BLIP3o-60k.

License

The PixelDense weights are released under the MIT License. They are fine-tuned from PixelGen-XXL T2I, which is released under Apache-2.0.

Citation

@inproceedings{yang2026pixeldense,
  title={PixelDense: Dense Prediction as Representation Alignment for Pixel Diffusion},
  author={Yang, Lehan and Qi, Daiqing and Zhang, Wenhao and Li, Avery and Yang, Yiqing and Li, Yifan and Kong, Yu and Zheng, Haitian and Zhang, Zhifei and Lin, Zhe and Jampani, Varun and Li, Sheng},
  booktitle={Advances in Neural Information Processing Systems},
  year={2026}
}

Acknowledgement

PixelDense is built on PixelGen and DeCo. The teachers come from DINOv2, SAM 2, Depth Anything V2 and Metric3D.