4DAnyone: Create Anyone in 4D from a Casual Monocular Video

Project Page  |  Paper

4DAnyone turns a casual monocular video into multi-view videos, enabling downstream 4DGS reconstruction.

Installation

git clone https://github.com/ant-research/4DAnyone.git
cd 4DAnyone
git submodule update --init third_party/GVHMR

conda create -n 4danyone python=3.11 -y
conda activate 4danyone
pip install -r requirements.txt

For faster inference, optionally install FlashAttention-3 or SageAttention.

Missing models and examples are downloaded automatically on first use. You can also download them manually:

python scripts/download_smplx.py
python scripts/download_model.py
python scripts/download_example.py

Inference

4DAnyone supports flexible target-view counts, pitch layers, and yaw coverage. Here are several common camera configurations:

6-view full orbit

A compact 360° layout for basic coverage. Start here for an initial test.

python inference.py \
    --video_path "data/source/pexels/2785536-uhd_2160_3840_25fps.mp4" \
    --views_per_layer 6

24-view full orbit

A dense 360° layout with broad angular coverage, suitable for 4DGS reconstruction.

python inference.py \
    --video_path "data/source/pexels/2785536-uhd_2160_3840_25fps.mp4" \
    --views_per_layer 24

48-view, three pitch layers

This layout distributes views across three pitch rings for broader coverage, enabling free-viewpoint 4DGS rendering.

python inference.py \
    --video_path "data/source/pexels/2785536-uhd_2160_3840_25fps.mp4" \
    --views_per_layer 16 --layer_pitches '[-10,15,35]'

8-view frontal arc

A focused layout for applications that only require front-side viewpoints.

python inference.py \
    --video_path "data/source/pexels/2785536-uhd_2160_3840_25fps.mp4" \
    --views_per_layer 8 --start_yaw -90 --yaw_span 180

Arguments

Run python inference.py --help for the full list. Key camera-layout arguments are:

• views_per_layer: number of evenly spaced views per pitch layer; must be divisible by 4 or 6.
• layer_pitches: pitch angles in degrees, one per layer; positive values place cameras above the subject. Total views are views_per_layer × len(layer_pitches).
• start_yaw: horizontal angle of the first view, in degrees; yaw 0 is the front view.
• yaw_span: horizontal range covered by each camera layer, in degrees.

Output

With the default --data_dir data, results follow this layout. See the output documentation for the complete format.

data/
├── gvhmr/results/<clip>/          # reusable motion-recovery result
└── fdanyone/<clip>/
    ├── metadata.json              # run settings, timings, resources
    ├── cameras.json               # the final N-camera rig
    ├── skeletons/00.mp4 ... <N-1>.mp4
    └── videos/
        ├── sparse/{00,04,09,12,14,19}.mp4  # default 24-view RCP proposals
        └── dense/00.mp4 ... <N-1>.mp4       # generated target views

Custom data

Use an input video that:

• is 720p or higher, with 1080p recommended;
• uses a 9:16 portrait aspect ratio;
• shows one person in a full-body or upper-body shot;
• has at least 121 frames;
• contains only mild camera motion.

3DGS Reconstruction

Install Nerfstudio before running ns-train.

# Export frame 0 across all generated views
python scripts/export_nerfstudio.py \
    --result_dir data/fdanyone/<clip> \
    --frame_index 0

# Train foreground-only 3DGS
ns-train splatfacto \
    --data data/nerfstudio/<clip>/frame_000 \
    --pipeline.model.background-color random

Todos

• Low-memory inference (<32 GB)
• Faster inference with TensorRT and sparse attention
• Support 4DGS reconstruction with an open-source method

Citation

If you find 4DAnyone useful or interesting, please cite our work and consider giving the repository a star ⭐:

@article{jin2026fdanyone,
  title={4DAnyone: Create Anyone in 4D from a Casual Monocular Video},
  author={Jin, Yudong and Xie, Tao and Zhang, Qihang and Shen, Zehong and Xu, Zhen and Shen, Yujun and Bao, Hujun and Zhou, Xiaowei and Xu, Yinghao},
  journal={arXiv preprint arXiv:2608.20335},
  year={2026},
  url={https://arxiv.org/abs/2608.20335}
}