There is a growing demand for customized and expressive 3D characters with the emergence of AI agents and Metaverse, but creating 3D characters using traditional computer graphics tools is a complex and time-consuming task. To address these challenges, we propose a user-friendly framework named Make-A-Character (Mach) to create lifelike 3D avatars from text descriptions. The framework leverages the power of large language and vision models for textual intention understanding and intermediate image generation, followed by a series of human-oriented visual perception and 3D generation modules. Our system offers an intuitive approach for users to craft controllable, realistic, fully-realized 3D characters that meet their expectations within 2 minutes, while also enabling easy integration with existing CG pipeline for dynamic expressiveness.
Method
The overview of Make-A-Character. The framework utilizes the Large Language Model (LLM) to extract various facial attributes(e.g., face shape,
eyes shape, mouth shape, hairstyle and color, glasses type). These semantic attributes are then mapped to
corresponding visual clues, which in turn guide the generation of reference portrait image using Stable
Diffusion along with ControlNet. Through a series of 2D face parsing and 3D generation modules, the mesh
and textures of the target face are generated and assembled along with additional matched accessories.
The parameterized representation enable easy animation of the generated 3D avatar.
Features
Controllable
Our system empowers users with the ability to customize detailed facial features, including the shape of the face, eyes, the color of the iris, hairstyles and colors, types of eyebrows, mouths, and noses, as well as the addition of wrinkles and freckles. This customization is facilitated by intuitive text prompts, offering a user-friendly interface for personalized character creation.
Highly-Realistic
The characters are generated based on a collected dataset of real human scans. Additionally, their hairs are built as strands rather than meshes. The characters are rendered using PBR (Physically Based Rendering) techniques in Unreal Engine, which is renowned for its high-quality real-time rendering capabilities.
Fully-Completed
Each character we create is a complete model, including eyes, tongue, teeth, a full body, and garments. This holistic approach ensures that our characters are ready for immediate use in a variety of situations without the need for additional modeling.
Animatable
Our characters are equipped with sophisticated skeletal rigs, allowing them to support standard animations. This contributes to their lifelike appearance and enhances their versatility for various dynamic scenarios.
Industry-Compatible
Our method utilizes explicit 3D representation, ensuring seamless integration with standard CG pipelines employed in the game and film industries.
Video
-->
Created Characters & Prompts
Make-A-Character supports both English and Chinese prompts.
A young，cute Asian woman, with a round doll-like face, thin lips, and black, double ponytail hairstyle.
A cool girl, sporting ear-length short hair, freckles on her cheek.
A girl with deep red lips, blue eyes, and a purple bob haircut.
A man with single eyelids, straight eyebrows, brown hair, and a mole on his face.
An Asian lady, with an oval face, thick lips, black hair that reaches her shoulders, she has slender and neatly trimmed eyebrows.
An old man with wrinkles on his face, he has gray hair.
A chubby lady with round face.
A boy with brown skin and black glasses, green hair.
-->
Make-a-Character + Audio-driven Talking
The generated characters can be driven by audio.
BibTeX
@article{ren2023makeacharacter,
title={Make-A-Character: High Quality Text-to-3D Character Generation within Minutes},
author={Jianqiang Ren and Chao He and Lin Liu and Jiahao Chen and Yutong Wang and Yafei Song and Jianfang Li and Tangli Xue and Siqi Hu and Tao Chen and Kunkun Zheng and Jianjing Xiang and Liefeng Bo},
year={2023},
journal = {arXiv preprint arXiv:2312.15430}
}
