CharacterShot: Controllable and Consistent 4D Character Animation
Paper • 2508.07409 • Published • 39
import torch
from diffusers import DiffusionPipeline
from diffusers.utils import load_image, export_to_video
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("Gaojunyao/Character2D", dtype=torch.bfloat16, device_map="cuda")
pipe.to("cuda")
prompt = "A man with short gray hair plays a red electric guitar."
image = load_image(
"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png"
)
output = pipe(image=image, prompt=prompt).frames[0]
export_to_video(output, "output.mp4")CharacterShot: Controllable and Consistent 4D Character Animation
CharacterShot is a controllable and consistent 4D character animation framework that enables any individual designer to create dynamic 3D characters from a single reference character image and a 2D pose sequence.
- Paper: CharacterShot: Controllable and Consistent 4D Character Animation
- Code: GitHub Repository
- Authors: Junyao Gao, Jiaxing Li, Wenran Liu, Yanhong Zeng, Fei Shen, Kai Chen, Yanan Sun, Cairong Zhao
Introduction
CharacterShot begins by pretraining a powerful 2D character animation model based on a DiT-based image-to-video model (CogVideoX). It lifts the animation model from 2D to 3D through introducing dual-attention module together with camera prior to generate multi-view videos with spatial-temporal and spatial-view consistency. Finally, it employs a novel neighbor-constrained 4D gaussian splatting optimization on these multi-view videos, resulting in continuous and stable 4D character representations.
Citation
@article{gao2025charactershot,
title={CharacterShot: Controllable and Consistent 4D Character Animation},
author={Gao, Junyao and Li, Jiaxing and Liu, Wenran and Zeng, Yanhong and Shen, Fei and Chen, Kai and Sun, Yanan and Zhao, Cairong},
journal={arXiv preprint arXiv:2508.07409},
year={2025},
}
Acknowledgements
The code is built upon CogVideo.
- Downloads last month
- 4