RecGen3D: Reconstruction-Guided 3D Generation in a Shared Canonical Space

Model weights for RecGen3D (SIGGRAPH Asia 2026, Conference Papers).

RecGen3D turns a handful of unposed, background-removed photographs of an object into a triangular mesh. It couples a feed-forward multi-view reconstruction backbone with a native 3D diffusion generator inside a shared canonical space: the reconstruction branch supplies canonical geometric anchors, and the diffusion branch refines and completes them.

• Code: https://github.com/zsh523/RecGen3D
• Paper: https://arxiv.org/abs/2604.01479

Files

File	Size	Description
vggt_canonical.safetensors	3.9 GB	Stage-1 canonical VGGT — the reconstruction branch, repurposed to predict in object-centric canonical space
recgen3d.safetensors	157 MB	Stage-2 LoRA adapters over Hunyuan3D-Omni plus the conditioning point encoder (72 M parameters)

Both are stripped of optimizer state, so they are weights only.

These are not the whole model

RecGen3D builds on two upstream checkpoints, which the setup script fetches for you:

• facebook/VGGT-1B — the base reconstruction backbone.
• tencent/Hunyuan3D-Omni — the VecSet VAE, DiT backbone and conditioner.

recgen3d.safetensors is small because Stage 2 adapts Hunyuan3D-Omni through LoRA; it is not a standalone generator.

Usage

git clone https://github.com/zsh523/RecGen3D.git && cd RecGen3D
bash install.sh && conda activate recgen3d
bash scripts/download_weights.sh          # fetches this repo and both upstream models
bash scripts/run_examples.sh              # 33 bundled examples

Your own images — RGBA PNGs of one object, background removed, two to eight views:

python inference.py \
    --ckpt pretrained_weights/recgen3d/recgen3d.safetensors \
    --images path/to/view_*.png --name my_object --output outputs/

About 30 s and 19 GB of GPU memory per object on an RTX 4090.

Citation

@inproceedings{huang2026recgen3d,
  title     = {RecGen3D: Reconstruction-Guided 3D Generation in a Shared Canonical Space},
  author    = {Huang, Zhisheng and Chen, Jiahao and Lin, Cheng and Hu, Chenyu and Huang, Hanzhuo and Yu, Zhengming and Li, Mengfei and Liu, Yuheng and Gu, Zekai and Zhao, Zibo and Liu, Yuan and Li, Xin and Wang, Wenping},
  booktitle = {SIGGRAPH Asia 2026 Conference Papers},
  year      = {2026}
}

Acknowledgements

Built on VGGT and Hunyuan3D-Omni.