Learn to create a stunning 3D animation with AI! This guide shows how to combine video motion capture, text prompts, and transitions using DeepMotion tools.
Creating a multi-scene 3D animation might sound daunting, but with DeepMotion’s AI-powered tools, it becomes a seamless and efficient process. In this tutorial, we’ll walk through how to build a dynamic three-scene animation using two videos and one text prompt. The scenes include a Shaolin Kung Fu performance, a baseball pitch, and a basketball dunk.
We start by uploading our video references into Animate 3D, DeepMotion’s AI motion capture platform. For this project, we’ll use:
1. A video of a Shaolin Kung Fu performance.
2. A video of a baseball pitch.
• Drag and drop the Shaolin Kung Fu video into Animate 3D.
• Choose the child character to match the performer in the video.
• Leave the motion capture settings at their defaults, or adjust as needed.
• Once processed, preview the animation. The AI captures most of the intricate movements, even handling challenges like loose clothing and fast motions.
• Drag the baseball video into Animate 3D and create the animation.
• Review the motion to ensure smoothness and accuracy. Artifacts or issues, such as unnatural arm swings, can be refined later.
Refinements help eliminate any artifacts and enhance the natural flow of motion.
• Export the baseball motion to SayMotion.
• Use the Refine Tool with a creativity level of 0.3 to retain most of the original motion while improving details.
• Provide a text prompt like “A person is swinging their right arm to pitch the ball.” to guide the AI’s refinement process.
• Generate eight variants and select the best one based on accuracy and fluidity.
• Export the Shaolin animation to SayMotion.
• Use the Refine Tool with a creativity level of 0.1 for preserving most of the details from the original motion capture
• Provide a text prompt like “A person performs Shaolin-style martial arts” to guide the AI’s refinement process.
• Generate eight variants and select the best one based on accuracy and fluidity.
Combine the first two animations into a single sequence using the Merge Tool:
1. Add the Shaolin animation as the first piece.
2. Add the baseball animation as the second piece.
The AI will automatically create a smooth transition between the two scenes, ensuring seamless movement as the character moves from martial arts to the baseball pitch.
To complete the animation, we’ll use the Inpainting Tool to add a basketball dunk scene after the merged Shaolin and baseball animations:
1. Open the Inpainting Tool and add a new section, approximately two seconds long.
2. Slightly overlap this section with the end of the baseball animation to ensure a smooth transition.
3. Use the text prompt: “A person starts running toward the basketball hoop and performs a dunk.”
4. Push the Generate Animation button to create the dunking motion.
The AI will generate the dunk sequence and seamlessly connect it to the existing animations.
Once the animation is complete, you can export it in multiple formats:
• MP4 Video: Toggle on the MP4 Output option, select a background (e.g., green screen), adjust the camera angle, and click Generate to render the video. After the generation completes select the MP4 file from the pull down menu beside the Download button and push the Download button to download.
• GLB/FBX File: select the GLB/FBX format from the pull down menu beside the Download button and push the Download button to download the 3D animation as a GLB/FBX file for further editing in any Digital Content Creation (DCC) tool.
Here is a more elaborated video tutorial of the above process.
With DeepMotion’s Animate 3D and SayMotion, you can create complex, multi-scene 3D animations using a combination of video sources and natural language prompts. Whether you’re a seasoned animator or a creative enthusiast, these tools make 3D animation accessible and enjoyable.
