ChatGPT is a redhead

We asked ChatGPT what they would look like if they were human.

Prompted by: ChatGPT

Created by: Asa BaileyImage

by: Midjourney

visit https://aifilmcompany.com

VFX artists show that Hollywood can use AI to create, not exploit

The fears around AI in production are, not to say illusory, but certainly a bit misleading. Generative AI like image and text models have improved greatly, leading to worries that they will replace writers and artists. And certainly studio executives have floated harmful — and unrealistic — hopes of partly replacing writers and actors using AI tools. But AI has been present in film and TV for quite a while, performing important and artist-driven tasks.

One early example came in a pair of Pixar presentations at SIGGRAPH 2023 about animation techniques used in their latest film, Elemental. The characters in this movie are more abstract than others, and the prospect of making a person who is made of fire, water or air is no easy one. Imagine wrangling the fractal complexity of these substances into a body that can act and express itself clearly while still looking “real.”

As animators and effects coordinators explained one after another, procedural generation was core to the process, simulating and parameterizing the flames or waves or vapors that made up dozens of characters. Hand sculpting and animating every little wisp of flame or cloud that wafts off a character was never an option — this would be extremely tedious, labor-intensive and technical rather than creative work.

But as the presentations made clear, although they relied heavily on sims and sophisticated material shaders to create the desired effects, the artistic team and process were deeply intertwined with the engineering side. (They also collaborated with researchers at ETH Zurich for the purpose.)

Simplified example of NST in action adding style to Ember’s flames. Image Credits: Pixar

One example was the overall look of one of the main characters, Ember, who is made of flame. It wasn’t enough to simulate flames or tweak the colors or adjust the many dials to affect the outcome. Ultimately the flames needed to reflect the look the artist wanted, not just the way flames appear in real life. To that end they employed “volumetric neural style transfer” or NST; style transfer is a machine learning technique most will have experienced by, say, having a selfie changed to the style of Edvard Munch or the like.

“If anyone ever tells you that Pixar used AI to make Elemental, that’s wrong,” said Pixar’s Paul Kanyuk pointedly during the presentation. “We used volumetric NST to shape her silhouette edges.”

(To be clear, NST is a machine learning technique we would identify as falling under the AI umbrella, but the point Kanyuk was making is that it was used as a tool to achieve an artistic outcome — nothing was simply “made with AI.”)

Images from Nimona, which DNEG animated. Image Credits: DNEG

Artists using AI to multiply their efforts “enables dialogue between creators and directors,”

I heard a similar note from Martine Bertrand, senior AI researcher at DNEG, the VFX and post-production outfit that most recently animated the excellent and visually stunning Nimona. She explained that many existing effects and production pipelines are incredibly labor-intensive, in particular look development and environment design. (DNEG also did a presentation, “Where Proceduralism Meets Performance” that touches on these topics.)

“People don’t realize that there’s an enormous amount of time wasted in the creation process,” Bertrand told me. Working with a director to find the right look for a shot can take weeks per attempt, during which infrequent or bad communication often leads to those weeks of work being scrapped. It’s incredibly frustrating, she continued, and AI is a great way to accelerate this and other processes that are nowhere near final products, but simply exploratory and general.

Artists using AI to multiply their efforts “enables dialogue between creators and directors,” she said. Alien jungle, sure — but like this? Or like this? A mysterious cave, like this? Or like this? For a creator-led, visually complex story like Nimona, getting fast feedback is especially important. Wasting a week rendering a look that the director rejects a week later is a serious production delay.