XR for Animation: the Future is Less, Not More

Let’s say you want to animate a 3D character. In the digital age, it should be easy, right? Technology has advanced enough that you can use motion capture (mocap) to film an actor going through a character’s actions, then clean up the resulting animation, rather than manually animate every motion of a character with software. Seems simple from the outside, until you realize the amount of infrastructure you’re going to use. You’ll need a large amount of camera hardware to capture all the right angles, a dedicated studio with enough space for those cameras and mounting racks, and then you need a motion capture suit for the actor. You also need the right software, to read and modify the footage. Those tools, the studio time, the actor, and professional cleanup could cost you tens of thousands of dollars. 

How is that accessible to the everyday creator? 

It could be that producing 3D animation using traditional methods–be it for television, films, or game development–isn’t approachable yet. But XR advancements are yielding a boom of tools and programs to streamlining this kind of digital content creation. Especially now, when it isn’t safe for large groups of people to convene on massive sets, or when an indie developer wants to create captivating content from the safety of their home. XR and AI can offer an opportunity for creators of any size, ambition, or wallet to expand this kind of content creation in a way that democratizes 3D animation and keeps the safety of creators in a pandemic society front-of-mind. 

In XR–as we’ve seen with facial scanning software and other AR/MR applications–creators can take advantage of single cameras for full-body motion capture if they use intelligent (read: AI-based) scanning. For most, “single-camera” means using a phone or tablet. The modern smartphone may have limitations compared to a studio rig, but more consumers have phones in their pockets than the latest mocap setup. At my company  DeepMotion, we’ve created a full-body motion capture cloud service, Animate 3D, that uses our perceptive AI technology in order to turn 2D videos into 3D animations, while costing thousands of times less than what many leading multi-camera companies would charge. This allows greater creative flexibility for creators and projects of all scopes and sizes. There is a matching imperative need for accessible full-body capture in today’s digital production landscape. 

This kind of AI-driven motion capture also eliminates the need for almost the most expensive component next to the camera: the mocap suit. Not using restrictive suits, also referred to as “markerless motion capture,” means more spontaneous, authentic movement. Fusing single-camera capture minus uncomfortable mocap suits allow creators to eliminate time-consuming, expensive studios, or multi-camera studio services with major data clean-up from their production budgets. With perceptive AI, you can film or use footage with few limitations for the setting, for filmmaking in spaces that may not be a creator’s first choice for filming, or with access to troves of videos readily available on the internet–a great workaround when self-isolating at home. 

These advancements sound hypothetical, but XR and AI are rapidly advancing to democratize motion capture in cost-effective, flexible ways, right now. Individuals with a love of animation or game design, or small studios with tight budgets, have been hungry for an inexpensive alternative for creating 3D animations for some time, with current leading motion capture technologies being a luxury only major companies could afford. Thanks to the XR renaissance, however, a boom of technologies have emerged aimed at these underserved corners of the animation industry. As they improve to match the power and performance of their Hollywood-ready hardware counterparts, they stand to transform the entire animation industry as we know it. 

https://www.vrfocus.com/2020/12/xr-for-animation-the-future-is-less-not-more/