Lifting Casual Human Video into 4D
- Input video
- Generated view 00
- Generated view 08
- Generated view 16
- 4D Gaussian reconstruction
In the Wild
4DAnyone generalizes robustly to diverse human-centric footage, producing consistent target-view videos and high-quality 4DGS models from casual monocular videos (mild camera motion, unknown camera intrinsics and poses).
Source Video
Generated Target Videos
4DGS Rendering
Featured
Dance
Character
Sports
Stage
Speech
Fashion
Method
A photorealistic 4D human is normally reconstructed from a calibrated array of synchronised cameras, which severely limits real-world use. 4DAnyone instead creates one from a single casually captured video, by generating the videos that array would have recorded.
The hard part is consistency at reconstruction scale: training 4DGS needs tens of views, but limited computing resources force generation into groups, leading to cross-group structural drift. We address this on two fronts:
- Scalable multiview consistency: Reference Context Packing (RCP) compresses the linearly growing reference context into a fixed budget, and Target Context Routing (TCR) routes context across the disjoint groups to enable cross-group structural communication.
- Accuracy-over-density conditioning: A 3D skeleton supplies sparse-but-accurate geometric cues, a much more robust conditioning signal than fragile dense depth and noisy camera parameters, enabling solid in-the-wild generalization.
The source video yields a 3D skeleton that guides generation at every target view. Generated views feed back as RCP references, TCR routes the groups, and the final videos train a 4DGS model with FreeTimeGS.
Comparisons
4DAnyone outperforms prior SoTA methods in both novel-view video quality and downstream 4DGS reconstruction.
Acknowledgements
We would like to thank Sida Peng, Dingkun Wei, Haotong Lin, Shuaiting Li, Zelin Gao, Yifan Yang, Yifan Wang, and Taoming Liu for their insightful discussions.
BibTeX
@article{jin2026fdanyone,
title={4DAnyone: Create Anyone in 4D from a Casual Monocular Video},
author={Jin, Yudong and Xie, Tao and Zhang, Qihang and Shen, Zehong and Xu, Zhen and Shen, Yujun and Bao, Hujun and Zhou, Xiaowei and Xu, Yinghao},
journal={arXiv preprint arXiv:2608.20335},
year={2026},
url={https://arxiv.org/abs/2608.20335}
}