THE SORCERER’S APPRENTICE - REAL MAGIC

As Double Negative completed close to half of the visual effects shots in ‘The Sorcerer’s Apprentice’, about 385 in total. Adrian De Wet was in New York from December 2008 to participate in pre-production meetings at Steiner studios, the old Brooklyn Navy yard, and travelled between there and London up until shooting started in April 2009.

Wet and Wild
“The first section of the project we got started with, right from December 2008, was the previs for a very complex sequence called ‘Fantasia’, headed by Animation Supervisor James Lewis. Previs was extensive for many sequences through the film and most of it was handled by Steve Yamamoto’s company Unit 11, but we did our own for Fantasia,” said Adrian.

Recalling Mickey Mouse’s misadventures in the 1940 Walt Disney film, Fantasia required coordinating several different types of effects. “It was a huge exercise in water simulations, but all the props are CG as well so, in all, the work comprised a massive, coordinated simulation. If any element, like a broom in the background, had to be changed, the whole sequence would have to be redone. All the parts were interconnected. It put a lot of pressure on the team to finish in time.” The simulation was based on several software platforms including Maya particles, Double Negative’s own fluid application and fluid renderer and others.

The main water effect comes at the end when the sorcerer Balthazar enters the laboratory and disperses the massive flood of water. In a wide shot from a balcony overlooking the set and a medium close-up of the water dispersing, all the water is completely CG. Adrian focused on the creative side of the scene, including control of the water. How should it look? How should it be animated?

Squashing Splashes
The level of splashing, especially for the water dispersal, caused both a technical and creative challenge. It had to reveal the floor underneath and respond to Balthazar’s magical hand gestures. Balthazar himself obscured part of the view, which made it harder to communicate to the audience that he was causing it to drain away.

A technical problem arose because the huge volume of water that had gathered couldn’t simply be compressed and disappear. “Fluid simulation software, both off-the-shelf and our own, tries to obey the laws of physics. All that water had to go somewhere, which meant we had to cheat a bit and reduce its level fairly quickly to make it appear to disperse. In the closer shot, we needed to reveal the floor with its distinctive stone geometry, and created extra elements of the water running off along the grooves.

“My previous water work has been more about sprays and splashes than huge volumes. Fortunately, throughout the work on the sequence, the special effects team flooded the set with ankle deep water, so we had good reference for still water, and our job became churning it up effectively. I made sure, during a lunch break, to film the set being drained, shooting from up on a balcony. It was purely for our reference of course. In reality, draining the laboratory took over half an hour, not just a few moments.”

Green Puppeteers
Getting into the action part of Fantasia involved animating a troop of mops and sponges – all CG bar a few ‘live’ ones mixed in. Most shots were filmed in duplicate or triplicate because the team was using puppeteers in green suits to hold the mops and brooms, especially where the props needed to interact with Dave and especially when they were splashing in the foreground water, close up.

“Having the live action really helped with these moves,” Adrian said. “We would shoot one ‘hero’ performance take when Dave was fighting a mop being handled by a green screen puppeteer. Then, copying the same camera move, we would shoot the same scene as a clean plate to supply to background

We needed to replace the puppeteer. The cameramen are very skilled at precisely repeating these camera moves. Trying to use motion control would have taken far too long.

Mop Physiology
“After the editor Billy Goldenberg had cut the sequence, we assessed the shots and decided how many and which CG props we’d need to put in. I’d say about 90 per cent are CG.” Nevertheless, you can only make a real mop do a limited range of actions. Animation Supervisor James Lewis handled the rigging challenge of mop physiology, starting with storyboarding.

They decided that in order to give the mops personality and character without facial expression or even arms and legs, they would try bunching up the strings to act a bit like limbs. “It gave us options for movement and expression – they can wag like a tail, look angular and aggressive like a spider, creep on tip toe, or run around. There was more than we expected. The director stressed that they shouldn’t look ‘cartoony’. They should be characterful and expressive but maintain a mop’s rigidity and obey the laws of physics. The handle couldn’t simply bend, wobble and stretch. They couldn’t fly around – walk, jump, fight and play, but within limits.

Their brief from Director Jon Turteltaub was only a few lines of script, just basic story points, from which they built up a library of actions, gags and vignettes of personalities to develop their story from. “Our first iteration was about nine minutes, three times as long as required. From there we added, removed and revised different pieces through late April into principle photography. We’d hoped to go into production with a locked down sequence but it didn’t resolve itself so easily. On set, the director and editor also began thinking of changes to make.”

Inspiration
They shot extensive footage. The first unit shoots captured the shots involving Dave, the lead character. They returned a couple of weeks later for the 2nd unit VFX shoot, run by the overall VFX Supervisor John Nelson, to gather more detail around the Training Room, continuously collecting texture and lighting reference. They shot stills of the entire Training Room set, taken over two nights, and Lidar scanned it at high-resolution to have the option to completely rebuild it in 3D if required. This data became useful to use for the cleanup needed on many shots.

Much inspiration came from Disney’s original ‘Fantasia’, and they actually borrowed a few ideas. For example, like Mickey, Dave picks up an axe and tries to chop one of the mops up and fails, while his shadow wearing a hood looms on the wall behind, recalling the classic wizard hat.
Another animation task that challenged the team was the famous bronze bull in Wall Street. When the sorcerers arrive in New York, the story’s location, the city is filled with magic, including this massive statue. ‘We had to give it the characteristics of two quite different creatures. It had to keep its original statue persona but also have the ability to run and charge like a bull, looking mean, angry and muscular.

Limping Bull
“On inspection, we discovered that instead of an anatomically correct bull, the statue was a piece of expressionist art. The typical practice is to start with a Lidar scan of a model to be animated, reduce the number of polygons a bit to simplify it and then rig it to move the way you want. But this statue’s hooves are obliquely angled and the legs are longer one side, causing him to lean over. It could never have walked without limping. We had to re-design him from scratch.”

After scanning the statue, they not only re-built him to be able to run, they also needed to change his facial expression. “The sculpture has a smiling, laughing expression but the script called for a menacing face. Then we had to give him a solid metallic appearance with the huge weight of a solid brass bull. But in fact, preventing muscular compression and stretching of the skin left it looking like a tin robot with badly made joints. We watched videos online of bulls running at Pamplona, for example, and noticed their wide range of gaits, from a high elegant trot to lowering their heads to charge. We built a library of animation cycles for him.”

Black Slime
Double Negative was responsible for the movie’s final fiery battle. They did some of their own previs for this section and, again, the complete sequence comprised a number of smaller ones that needed expertise in various techniques. The team handed one scene to another VFX company, One of Us, while retaining creative control throughout. When the evil sorceress Morgana bursts from the Grimhold, the container in which the sorcerers have been trapped for centuries, the effect is fully CG.

The sorcerer Horvath places the Grimhold on top of a fountain filled with CG water, stops the fountain and magically cracks the container open with his cane. Black slime pours out, created as a fluid simulation by One of Us. It rises and grows, with CG fluids composited over the surface to give it the look of an egg sack holding all things evil writhing under the surface. When it finally bursts open and falls, we see Veronica standing there, possessed by Morgana.

A new sequence then begins as she begins casting fire over the city from the fountain. Double Negative focused on creating this fiery energy that travels through the streets. A helicopter unit took a collection of aerial views of Manhattan, which the team would composite the fire into, while on the ground, a camera car captured more shots, moving up Broadway and down side streets with a camera strapped to the front and a 20K light to create some interactive lighting as it drove past.

Fiery Energy
To decide on the looks and form this fire should take, they experimented for months – should they develop a fluid simulation, use actual fire or something magical and altogether different such as embers? It was referred to in the script as ‘fiery energy', which suggests something a bit more magical than just natural looking fire, although it did have to look real. At one point it was looking very subtle, almost invisible.

“It was a creative challenge more than anything else. Technically it was all within reach. We needed to establish the look of fiery lines and symbols in the sky that were readable in wide aerial shots as well as close ups where it bounced off satellite dishes. And because it was fiery energy rather than just fire, we wanted to avoid making it look explicitly like fire to help establish the story point that it was something magical that normal eyes wouldn't see – after all, this was the fire that was going to form the magic Pentagon over Manhattan.

“We ended up with a fireworks look, very hot, energetic and unstable. Senior FX TD Tim Riley developed a Houdini set up based on particles and sheets of geometry, with a leading fiery ‘head’ made out of combusting energy. Render times were relatively low, making it possible to quickly get a selection of elements into compositing and allow the look to develop further in the comp. This enabled the look of the fiery energy to evolve in the comp without the underlying setup needing to change, and without the need to re-simulate complex fluid dynamics.” They relied again on Squirt, their fire and fluid simulation software. Sequence supervisor Gavin Graham gave a presentation on Squirt at SIGGRAPH this year, just as this film was being released.

Spectral Morgana
Eventually, Morgana shows her true evil self, a disembodied soul appearing in a rush of particles. “The brief for this was mostly about what the director did NOT want,” Adrian said. “It wasn’t to be like anything we’d seen before, or like a traditional cinematic, ethereal ghostly being. It was to be firmly present in the image, not mixed on at 50 per cent or confined to itself as a regular CG creature. It had to shape-shift and change form, move by drifting and be able to dissipate and lose edges and regroup elsewhere. This form made her much harder for Dave to overcome and destroy.

“Nevertheless, she still had to be recognizable as the same character, played by Alice Krige, as earlier in the story with facial detail you could read and defined hands that you could follow. We shot Alice on set, meaning we could base our ‘spectral Morgana’ on complete plates of her real performance, not on character animation. But this also demanded very accurate body tracking, including a cloth rig track to accommodate her long trailing black dress and sleeves, and facial animation tracking, requiring a facial rig. This gave us a fully rigged digital double of Alice doing exactly what the live footage was doing but allowing us to turn her into particles.”

In the early concept drawings, Morgana appeared as a hideous witch. But the director and producer had liked Alice Krige’s performance so much that they wanted the team to both give her an otherworldly quality to emphasise her magical powers, while maintaining the readability of her performance.

Particle Management
This meant using as much of the original plate as possible. During look development, swirling particles were developed and a set-up was created in Houdini, beginning with a volumetric representation of her geometry and using curl noise and fluid forces to give a constantly shifting form. The difficulty was adding just the right amount of movement to avoid losing her altogether, so they layered the particle simulation. The tightest layer showed her shape more clearly and looser layers revealed more movement. They reprojected the plate onto the particles, adding retexturing controls.

But using particles alone didn’t give the character enough complexity and density. When the particles separated, you could see though Morgana. Elements had to be added to the interior. First, a shining, pulsing core was placed in her centre by raytracing against a fluid simulation. Then a swarm of writhing fishlike creatures was created and also set up using the procedural framework of Houdini, to be able to apply it across a large number of shots.

2D Sequence Co-Supervisor Ian Simpson was responsible for putting these various components together, using Shake to composite her into the scene. Interactive LEDs on Morgana’s hands in some of the shots, representing the plasma she was shooting, complicated the work. Depending on the timing of the plasma balls, he had to reconstruct a clean plate through the CG to prevent the blue plasma from interfering with her form. Getting the right balance between swirling and tighter layers of particles, especially on the face, was still a problem, requiring many masks, texture and lighting passes, and she also needed to be scaled in 3D in many shots.

Real Physics
Magic notwithstanding, Jon Turteltaub always wanted the effects to be based on real physical principles, including Balthazar’s training sequences with Dave. When he taught his apprentice how to throw fire balls for example, the ball had to appear composed of real flames, not stylised fire. When the plates were shot, the team wanted to introduce real interactive lighting for the balls, and the lighting gaffer Michael Gallart helped them out with small bright LEDs that could be concealed in the actor’s hand or put on an elastic band rig between both hands, and would cast a light on their faces.

“These worked out well. We had a ‘cool’ toned light for the fire balls and a hotter blue light for the plasma balls introduced later. Actually, the light on this set was mostly real, which is essential to properly integrating the CG elements, and only needed a little enhancing later. We also discussed working with the fireballs with the actors to make sure the performance and the effects were aiming for the same looks, and because the director continued getting new ideas on set, this sequence became more of a post-vis exercise than previs.”

Balthazar’s magical demonstrations are perhaps at their most potent when he scorches a huge fiery circle into the stone floor of the laboratory. On set, the shoot required one of the biggest special effects rigs of the show, using gas plumbing under the floor to form the magic symbols. These could produce fire of any colour and green was chosen.

Balthazar stood at the centre, raised his arms and produced a huge rush of flames. Then the scene was shot again without Nick. “It looked quite effective, but needed some embellishment – the rig was smoky and although the flames were bright enough to blow out the frame, they weren’t really tall enough. The team eventually decided to replace the flames with CG but what really helped us was having the interactive lighting already baked into the plates. It was virtually all there for us.
“However, getting the fire to look right needed a surprising amount of work and a large simulation. While Balthazar is etching the symbols into the floor with hand motions, we also needed to show how much effort this demanded from him, and the heat and energy involved, which we did by making the animation stick or judder slightly instead of looking smooth.

“Looking at reference material of oxy-acetylene welding was useful because these flames are hot, flickering, super-bright, and blow out the frame with lens flares and reflections. It was very hard to emulate and took a lot of trial and error – mostly error,” Adrian said. “Fire remains a challenge, although at Double Negative we’ve developed solid systems for producing it and have worked with fire on many projects like ‘Harry Potter and the Half-Blood Prince’, Hell Boy II’ and ‘2012’. But this fire had a very stylised look. Being green put it into a cool colour palette as well, toning down the heat we needed to portray.”

Unfolding Book
A puzzling effect the team wrestled with was Balthazar’s magic book, unfolding repeatedly from a tiny prop into a weighty tome. Except for the original prop, the book is all CG. The surrogate object that actor Nicholas Cage is unfolding was a sheet of black wrap, a matte black aluminium material normally used for masking light leaks and reflections. It can be easily and quickly moulded to different shapes. Using a piece the size of the final book, they folded it down to the tiny size to begin with, and Balthazar unfolds this on the shoot.

“We expected this to be a pretty straightforward method but the material kept getting bent and warped and lacked the thickness of the book, of course. Oddly, it also proved very tricky to work out which side to open it from to make a logical progression through the scene. Consequently, we had to do some animation to get his hands into the right position, at times cleaning out his hands altogether and putting them back correctly. The weightiness of the full book was also missing from the live action. It took the animators weeks of comping and

Hungarian quick rug
An interesting effect that Double Negative worked on with an outside visual effects group, Ghost VFX in Copenhagen, was a ‘Hungarian quick rug’. Balthazar was to walk into the centre of the room where it lay, causing it to take on the properties of quicksand. His feet would begin sinking into it, pulling him further in the more he struggled.

They considered different approaches. Would it remain a surface, becoming a sheet pulled through a hole, pulling him with it? Should it become a set of particles on the surface that flow into the centre at different rates, distorting the pattern of the rug, or could it develop concentric ripples and bumps extending out from him? They chose this last option as most clearly illustrating the danger and jeopardy of the scene.

“The others would have looked good but not have delivered this message as well. The work on this effect required a number of simulations including a combined cloth and fluid simulation, and sometimes a particle sim. It needed to have a rug texture and wave like cloth, but also have fluid motion,” Adrian said.

The special effects team rigged up a tank on set full of water in the middle of which was a small platform on hydraulics which could be lowered into the water as required. Over the surface of the water, a latex membrane was placed and attached to the platform. So, when Balthazar stood on the platform as it was lowered down, the membrane would contain the surrounding water while it closed in around his legs, appearing to suck them in and down. The team then tracked a rug texture and pattern onto the membrane as it rose up around his body, revealing ripples and swells to add a fluid feeling. Finally, because the sequence was cut together differently to the order in which it was shot, creating continuity problems, the surrounding furniture was all replaced with CG for clean-up.

Lightning Bolts
A major story point for ‘The Sorcerer’s Apprentice’ is the lead character Dave’s own version of magic, his giant Tesla coils project. To settle on an approach for creating the huge bolts of electricity they emit, the team looked at as much reference as possible. The crew also hired a small coil to bring on set and shot it on film at different frame rates and shutter angles as it fired off. “We could analyse and break down the looks and behaviour of these ‘bolts of lightning’, which is essentially what the coils produce. After capturing their constituent parts, we tried to create a mathematical model for them and the geometry of their curve. Senior Effects TD Michael Parker used this information to create them in Houdini, partly with the L-systems plug-in, allowing us to specify a source and destination point and fire a lightning bolt between them.