Lifting Casual Human Video into 4D
- Input video
- Generated view 00
- Generated view 08
- Generated view 16
- 4D Gaussian reconstruction
In the Wild
4DAnyone generalizes robustly to diverse human-centric footage, producing consistent target-view videos and high-quality 4DGS models from casual monocular videos (mild camera motion, unknown camera intrinsics and poses).
Featured
Dance
Character
Sports
Stage
Speech
Fashion
Robot
Method
A photorealistic 4D human is normally reconstructed from a calibrated array of synchronised cameras, which severely limits real-world use. 4DAnyone instead creates one from a single casually captured video, by generating the videos that array would have recorded.
The hard part is consistency at reconstruction scale: training 4DGS needs tens of views, but limited computing resources force generation into groups, leading to cross-group structural drift. We address this on two fronts:
- Scalable multiview consistency: Reference Context Packing (RCP) compresses the linearly growing reference context into a fixed budget, and Target Context Routing (TCR) routes context across the disjoint groups to enable cross-group structural communication.
- Robust geometry conditioning: For generative models, conditioning accuracy matters far more than density. A 3D skeleton provides sparse yet accurate geometric cues without relying on fragile metric-depth estimates or noisy camera parameters, enabling strong in-the-wild generalization.
The source video yields a 3D skeleton that guides generation at every target view. Generated views feed back as RCP references, TCR routes the groups, and the final videos train a 4DGS model with FreeTimeGS.
Comparisons
4DAnyone outperforms prior SoTA methods in both novel-view video quality and downstream 4DGS reconstruction.
Acknowledgements
We would like to thank Sida Peng, Dingkun Wei, Haotong Lin, Shuaiting Li, Zelin Gao, Yifan Yang, Yifan Wang, and Taoming Liu for their insightful discussions.
We also thank Pablo Vela for his insights on inference acceleration and for building the 4DAnyone–Rerun Hugging Face Space, and James C. Kane for building a 4DGS player for 4DAnyone. We are working to make 4DAnyone faster and easier to use on consumer-grade hardware.
BibTeX
@article{jin2026fdanyone,
title={4DAnyone: Create Anyone in 4D from a Casual Monocular Video},
author={Jin, Yudong and Xie, Tao and Zhang, Qihang and Shen, Zehong and Xu, Zhen and Shen, Yujun and Bao, Hujun and Zhou, Xiaowei and Xu, Yinghao},
journal={arXiv preprint arXiv:2608.20335},
year={2026},
url={https://arxiv.org/abs/2608.20335}
}