How ‘Avatar: The Way of Water’ Solved the Problem of Computer-Generated H2O

For the original “Avatar,” the writer-director James Cameron made extensive use of cutting-edge performance capture technology. It enabled his human cast to portray the 10-foot-tall, pointy-eared, blue-skinned aliens called the Na’vi, the inhabitants of a jungle moon named Pandora.

These were groundbreaking visual effects in a filmmaking career with no shortage of them, from the shimmering water-tentacle in “The Abyss” to the shape-shifting liquid metal assassin of “Terminator 2: Judgment Day” and a largely computer-generated ship, populated by computer-generated passengers, in “Titanic.”

With his long-awaited follow-up to “Avatar,” Cameron set about exploring more of Pandora. The screenplay, which he wrote with Rick Jaffa and Amanda Silver, described many scenes in the water and underwater with a semiaquatic Na’vi clan called the Metkayina.

When it came to realizing those watery scenes, there would be none of the usual Hollywood “dry-for-wet” performance-capture techniques: actors dangling from wires, feigning weightlessness, making vague swimming motions in the air. According to members of Cameron’s crew, the director insisted on “wet-for-wet.”

“It’s about the credibility of the actor’s performance,” said Richie Baneham, a visual effects supervisor with Cameron’s production company Lightstorm Entertainment. “If an actor is genuinely in water, there’s a viscous resistance. It informs the actor’s choices. That’s what we’re chasing. That’s what makes it feel real.”

“Avatar: The Way of Water,” opening Friday, represents a new milestone in the evolution of visual effects technology: underwater performance capture.

After some crude tests — the first experiments took place in the backyard pool of an “Avatar” producer, Jon Landau — a performance capture tank was constructed at Lightstorm Entertainment’s facility in Manhattan Beach, Calif. The tank was 32 feet deep and held around 90,000 gallons. (By contrast, the seawater tank constructed for “Titanic” contained 17 million gallons.) Along with viewing platforms on the deck, there were windows in the pool walls for camera operators to shoot through, giving the place the look and feel of a laboratory aquarium.

At Cameron’s behest, the tank simulated waves and currents, too. “Oh, God,” Baneham recalls saying to his colleagues, “it just got more complicated, lads.”

One of the main difficulties the crew faced was preventing overhead studio lights from interfering with performance-capture data. For this, Cameron suggested spreading a layer of small polymer balls across the water line, thereby diffusing the light in the tank while allowing actors to surface safely. He had used the basic technique to simulate dark oceanic depths in “The Abyss.”

Something else that would have inconveniently disrupted the performance capture: actors with their pesky respiratory needs. Cast members held their breath, not just during takes but in the moments leading up to a take. On top of getting their scuba certification, they trained with a freediving instructor, Kirk Krack. Kate Winslet proudly holds the cast freediving record: seven minutes and 14 seconds.

For the occasional propulsive boost in the water, performers wore jetpack-like devices. There were safety divers on hand, and hot tubs for the actors between takes. The main cast also share their scenes with professional underwater dancers and gymnasts.

Reams of performance-capture data were gathered this way. For the next step in the process, that data was shared with the artists at Weta FX, the visual effects company that Peter Jackson co-founded in Wellington, New Zealand.

It’s the artists at Weta who transformed the wetsuited performers into the Na’vi. They also created the lushly detailed digital environments, transporting the action from a chlorinated tank to an enchanting underwater realm, replete with flora and fauna. According to Weta, 57 new species of sea creatures were created for the film. Weta artists also consulted with researchers at Victoria University of Wellington about coral reef biology. “We don’t get a free trip to the Bahamas,” a Weta effects supervisor, Joe Letteri, joked.

“Avatar: The Way of Water” is the biggest visual effects project the company has ever taken on. Only two shots in the entire film contain no visual effects.

Despite all those long skin-pruning sessions in the tank and the vast quantities of water in the sequel, nearly all of the water in the film is computer-generated. Of the 3,240 visual effects shots Weta worked on, 2,225 involved water.

To bring Pandora to life, the team at Weta had to become expert not only in hydrodynamics, but also in rendering those complex physics photorealistically. The way water splashes, sloshes, sprays and saturates — in the parlance of the industry — had to be “solved.”

“There were something like 1,600 different major effects simulations,” said Eric Saindon, another effects supervisor at Weta. “The proper flow of waves on the ocean, waves interacting with characters, waves interacting with environments, the thin film of water that runs down the skin, the way hair behaves when it’s wet, the index of refraction of light underwater. We wanted to make it all physically accurate.”

As part of their research, the team shot hundreds of hours of reference footage: wind ripples on the surface of water, waves hitting rocks, the movement of seaweed. Saindon enlisted a friend to get dunked on camera so that the team could study the effects of water on curly hair. “Our pool was not the warmest thing in the world. I don’t think she’d do it again for us,” he said. Cameron himself provided reference material from his own deep-sea dives.

Weta pioneered ways to make things look wetter, recently filing a patent application for “methods of generating visual representations of a collision between an object and a fluid.” This innovation was especially useful in one sequence where a human character, named Spider, emerges onto some rocks by a group of Na’vi, dripping with water. The shots blend live-action footage of the actor Jack Champion, who was filmed in a wave pool, with the CG Na’vi.

As Saindon pointed out, the simulated water on the Na’vi had to look just as convincing as the actual water on the human character “because you’re comparing it to a live-action element in the same shot. And we don’t want you thinking about that.”

Considerable horsepower is required, too. For that scene alone, it took two weeks for Weta’s systems to simulate just the movement of the water. That’s before the millions of processor hours required to render the graphics. (The total amount of data stored for this film was 18.5 petabytes, while “Avatar” required one petabyte.)

Another sequence features boats crashing through entirely simulated waves. “Everyone will just assume we were out on the open ocean,” Saindon said.

While the results are in a sense spectacular, the point of this kind of painstaking work is not to draw attention to itself. “It’s not about crazy moments that yell out ‘visual effects.’ These are just effects that make the film work,” he said. “We just want people to watch the film and be swept away, and forget that we did anything.”