VFX supervisors Arundi Asregadoo and Axel Bonami at MPC talk about new FX their global team developed while completing over 1,000 shots for action adventure movie ‘Ghost in the Shell’.
VFX supervisors Arundi Asregadoo and Axel Bonami at MPC led a global visual effects team to complete more than 1,000 shots for the action adventure movie ‘Ghost in the Shell’. They got started on the project in April 2016. In an interview with Digital Media World, Axel and Arundi said that by that time, production VFX supervisors Guillaume Rocheron and John Dystra had been working on the project for nearly two years, and the concepts, looks and set designs were well-developed.
The influence of the story’s Manga heritage on the visual style could already be seen across the production, but the director Rupert Sanders was also keen to find new looks for the production, and to present the audience with something original.
The city that the lead character Major inhabits and walks through, in daytime and night time scenes, avoids a polished futuristic look. It is a little dirty, full of life and people and is based on real places. Its importance to the story and to Major herself is evident in the extreme detail seen from every camera angle, and in the variety of elements and lights that constantly interact and change. The colour palette also references Manga looks and mood - colourful and bright but cool, without overpowering the action.
City Life
When we see the characters walking through the set, they are usually on real streets, not a green screen stage. Reference for the city comes mainly from stills and helicopter shots of Hong Kong with other photo reference from Shanghai and Tokyo. Starting with this real, physical base, MPC’s team built up the beautiful CG city audiences see in ‘Ghost in the Shell’.
To help orient the viewer within the city environment, five CG ‘ghost cam’ camera moves were created over the cityscape that serve as establishing shots, sometimes lasting as long as one and a half minutes. “A first pass was created from Google data at the pre-vis stage to decide the path of the ghost cam during the aerial shoot over the Hong Kong location,” Axel Bonami said. “To support the video, MPC’s on-set photographers captured photo reference along the path of the camera from rooftops, gathering the necessary reference material.
“The futuristic buildings seen within the environments were built as physical scale models, and then captured as photogrammetric data and digitized to 3D models. Shots were then fully computer generated using a huge library of assets, from the buildings to elevated highways, from street dressing to holographic street signage, adding hundreds of people, vehicles and machines. Because viewers frequently go with Major onto the roofs of the city’s towering buildings, as well as down at street level, the photo reference was especially important.”
But in fact, physical sets and environments were favoured generally in this project. Axel and Arundi described the sets as nearly complete and well-built so that the team could concentrate more on looks and effects and not so much on full 3D builds as on some other action films MPC has worked on recently.
Composition Control
Everywhere the camera looks, it was important to maintain both a coherent composition and consistent attention to fine detail, whether real or digital. Looks were sometimes more stylized than photoreal, and so the key lighting, colour palette and temperature were carefully controlled. Illumination from hundreds of complex, animated holograms is essential to the scenes, inside and out, but couldn’t be allowed to spoil the compositions.
Arundi said, “Advertising signage is a large element of the city’s special look, and takes the form not only of the holograms but also huge animated ‘solograms’. In contrast to holograms that produce their own light, the solograms are photoreal, interactive volumetric displays in the form of stylized people, used like moving 3D billboards in the city.”
To create these, the production’s team built a special custom rig holding 80 2K cameras to capture volumetric footage of actors. Thus each frame was represented at numerous angles, enough to produce photogrammetric data used to solve 32,000 3D scans for these elements. MPC’s R&D team developed the software the artists used to reconstruct, manipulate and eventually composite the volumetric data into shots.
New Look, New Language
The interactive qualities and high level of detail in the environments were carried into the interiors where the light is especially reactive to the action taking place, often featuring in the night club scenes.
In support of the director’s search for an original style, MPC developed some signature effects and uses of computer graphics for ‘Ghost in the Shell’. For example, many of the distinctive character effects in the film are based on voxelisation processes. It became a language for the artists, expressed as different effects with variations in the render, or in composting in Nuke.
“We made multiple uses of voxelisation, generated using Houdini,” said Axel. “MPC used it on the sologram builds, for example, to create a 3-dimensional grid revealing the character geometry resulting from the photogrammetry of the real, lit actors. The grid gave us control over density, resolution – that is, number of pixels - and glitches.” Digital glitches are another of the film’s signature effects. Both voxelisation and glitches are described in more detail through several examples, further on in this article.
Major Inside Out
Major’s body is completely robotic except for the living human brain inside her head, although her memories of the events leading to that state are fleeting and imperfect. As she looks out at the people around her and considers that they all feel connected to something while she does not, viewers can see that this isn’t actually true. She is connected to what is happening to her and to the city itself in a physical way that affects her appearance from scene to scene. Her interactive nature owes largely to her thermoptic skin, the ‘shell’ covering her internal robotic structure.
In the top image, Major wears her silica suit on set. Below, she wears the digital skin on screen.
Weta Workshops designed and made a practical suit of silica material that precisely fits the actress Scarlett Johansson for use on set. It was useful to have in camera to better understand the concept of a body-in-a-shell, but in the end the production decided that the suit was too thick and bulky for the look they were after. MPC replaced it digitally, based on scans and photography of Major, to appear thinner and more flexible, as though it is a real skin.
The audience witnesses the ‘shelling’ process her body went through to end up with her shell and looks she has in the film, beginning with her internal skeletal structure, the layers of muscles and the outer skin. “For this shelling sequence, the pre-vis was done by The Mill, and the skeletons were physical builds from Weta Workshops, used as a guide to framing the photography, whether the individual shots were entirely or partly digital,” said Axel.
“MPC then computer generated the skeleton to work with the required animation, muscle flex, reframing, composition, lighting and integration. Some of shots used the practical skeleton within a CG environment and particulate effects, and transitioned to the full CG version for more complicated shots. MPC’s work also includes the full CG Major and environment as she rises up through the different stages and chambers, up to the final reveal. The whole sequence was a true combined creative effort.”
Expressive Skin
Major’s expressive shell features a number of light effects including reflection, refraction and transparency. These effects also enhance her invisible state and emotional states when she heats up, both of which had to be visualised.
Her invisibility is critical to the story, her ‘ghost’ nature and her ability to fight. For MPC, it was a primary use of the voxelisation process. Axel said, “Generated in Houdini and rendered in Maya, we worked in collaboration with Guillaume and John using a similar language of voxels for the invisibility. The voxelisation allowed us to generate and control a 3-dimensional voxel grid of small cube pixels that could be activated by proximity to other objects. We would control its size, life span and ability to introduce digital artefacts around the subject.
“On screen, Major appears to produce a field around her of 3-dimensional pixels that can then show different optical responses - reflectivity, refractivity, self occlusion and opacity. We would generate maps for the different panels of the suit so that we could then play around with the different stages through panel glitches. So the invisibility becomes a combination of reflective, refractive, opaque, fully transparent and colour compression artefacts, using velocity maps to combine the whole.”
From the cubes, with normals, and a motion vector pass storing the direction for each pixel, they built up the invisibility and other looks for the suit with distortions. That is, by combining the directional data with compositing software [Nuke] that can read this data, it could be used to drive the actual amount of motion blur or other distortions for the shell to express invisibility, heat or emotions. The vector passes could be included in the render pipeline.
Seven Ways
Perhaps the most characteristic of her shell effects are ‘glitches’. Major sometimes refers to her emotional states as ‘glitches’ in her memory. In fact, ‘glitch art’ is a key effect in the film, and is seen in other characters and across the environments.
Arundi said, “The director requested our team to develop seven ways to describe a glitch - effects showing gaps and interruptions in data defining anything from the solograms’ occasional breakups, to Major’s flashbacks to her past, to the disintegration of holographic characters when they enter or leave a scene. Based on the voxelisation language, and due to the holographic or digital nature of the characters and elements, glitches were subtly added to the actors, using Nuke.”
Unstable Memories
Major’s ‘deep dive’ into the mind of a destroyed Geisha robot is full of still more voxelisation and glitch effects that, in this case, are used to conceptually describe data - specifically, data that has been interfered with. The director had some very clear ideas he wanted to illustrate in particular ways, again keeping as much of the action in camera as possible in spite of the fantastic nature of the environment and what was happening.
Major’s journey into the Geisha robot’s mind is a dangerous process because its memory has been hacked by the story’s dark, mysterious character, Kuze. The robot is taken into a kind of post mortem lab where its cable is inserted into Major. Viewers can watch her data as it travels into the Geisha bot. The environment inside its head was designed to give an organic nature to its coding - not simply to invent an abstract zeros-and-ones region in space.
Everything here is breaking down into its constituent data particles, visualized using the 3D voxelisation process. When Major walks into a night club set, built at the New Zealand location, she is walking into a world of unstable data memory – where recently recorded events are halted but unstable and gradually eroding. “What you are seeing is a combination of Houdini, with Nuke for secondary particulates and glitch effects,” said Axel.
Major finds Kuze the hacker there and sees him turning the Geisha into a sleeper-cell, hacking droid by installing a hacking device capable of stealing data, changing her initial submissive nature to a predatory one. As Major moves toward Kuze and attracts his attention, it throws her back into an agitated pile of data in which black figures representing malware chase her. To maintain the human, organic quality, 50 stuntmen covered in black slime grabbed and pulled her down.
At the same time as this is happening down within the Geisha’s internal world, on the surface in the lab Major becomes highly agitated physically.
Unfinished Body
One of most dramatic parts of the story is the relationship between Kuze, the hacker, and Major. Before they were built into robotic hybrids, they had known each other as human beings. “Kuze’s robotic body is an unfinished version of Major’s, and so similarities between them exist in their skeletons and muscle build, the original concepts for which were designed by concept artist Vitaly Bulgarov,” Axel said.
“Many of Kuze’s internal parts are exposed, allowing you to see the intricacies of the skeleton and muscles. Much of this detail he shares with Major but unlike her, his largely damaged exterior was made of small porcelain body panels, now illustrated with details of events related to Major, whose human name was Motoko.
“In several places, his body is so broken that the light passes through the internal structure completely and made him difficult to animate. It meant tracking the actor’s movements precisely while replacing limbs with the CG structures and the background detail.” The actor’s face could be retained throughout his sequences.
The director’s preference for physically based effects also applied to some of the character effects. When Major feeds a cable into the ports at the back of her neck, for instance, he preferred to use a closeup of a mannequin to make the insertions more believable. The laboratory doctor’s eyes that slide up and give way to a specialized mechanical pair are a practical build by Weta Workshop, digitally augmented in post.
For certain shots, the camera crew used a 21mm lens to produce, in-camera, a typical anime-style forced perspective image composition – that is, with an out-sized primary figure in the foreground while anyone or anything else becomes dwarfed in the background.
Collaboration
R&D for the project included work with software company Autodesk while defining the voxelisation and glitch processes. Other work was devoted to data and asset management, owing to the huge amount of detail and elements that had to be properly managed to maintain the controlled effect of the overall look.
“Between MPC’s Montreal, London and Bangalore facilities, about 600 artists worked on the project globally. The Bangalore team was a huge help in getting the modelling and compositing finished. Some 100 shots were completed in the last week alone. Our global team produced close to 80 minutes of VFX shots before the cut,” said Arundi.
“In spite of the emphasis on an in-camera approach to effects, our work includes over 70 full CG shots. For instance, almost 80 per cent of the battle sequence with the Spider tank had to be digitally replaced, but even this scene had a physical basis. It required destruction on a massive scale, created with MPC’s proprietary destruction software Kali.” www.moving-picture.com
Words: Adriene Hurst
Images: Courtesy of Paramount Pictures
