Awards Entries
Newsletter

THE THREE MUSKETEERS

Article Index

Page 1 of 2

Mr X met a formidable visual FX challenge in ‘The Three Musketeers’, building assets with millions of polygons, full CG shots in stereo 3D and perfecting light and textures on water and digital doubles. Digital FX Supervisor Eric Robinson and Lighting Supervisor Trey Harrell explain the details.

Digital Effects Supervisor Eric Robinson estimates that the complexity of many of Mr X’s full CG shots in ‘The Three Musketeers’ was greater than on any project their team has tackled before. Such shots involved recreating 17th century Paris, a hero version of Notre Dame for enormous battling airships to land on or crash into, and digital doubles performing on rooftops. Mr X’s full staff across all teams numbered over 100, and the shot count reached 274.

Previs and Post-vis
Mr X, based in Toronto, has completed several projects with director Paul Anderson, including the ‘Resident Evil’ films, and with Constantin Films, and has developed a sound relationship with Paul and the producers. Thus, they were involved with this new project from the early script reading stages and completed the movie’s previs. From there they moved to on-set supervision and started working with the editors on critical sequences, providing post-vis animatics to help stage the action correctly.
“Previs served two roles,” Eric said. “One was artistic giving the director a chance to plot out his vision and visualize the script beyond storyboards, but the other was technical. Some of the film’s more dramatic camera moves, for example, could be planned in previs. In a shot where the hero d’Artagnan is entering Paris along a bridge, from a location shoot in Bavaria, the crew was using a 50ft Technocrane, which Mr X built in Maya to determine how it could move to give desired coverage within a very limited space on the bridge. Paul would sometimes work with the animators designing cameras moves, which might later be enhanced with post moves in the composite.”

City Views
No production shooting was done in Paris, only research and texture shoots. Historical maps and illustrations were used to re-create the 17th century city, with modern maps and Google to fill in the gaps. “Initially, the majority of our Paris backgrounds were intended to be 2.5D digital matte paintings, but during production the camera moves grew increasingly aggressive. At one point during spring 2011, we realized that we needed to re-think our approach. Eventually, most of Paris ended up as matte painting assets baked down onto 3D geometry and lit in our sequence lighting rigs - or as true 3D matte paintings,” Eric said.

The magnificent airships, which according to the story were based on Leonardo da Vinci’s own blueprints, were among the team’s top CG challenges. As they sailed overhead, they were almost always viewed from below. The larger ship’s crow’s nest area is only viewed from above once. The team’s first task was to sell the height and scale of the ships, 300 to 400 feet above ground at any time. “This became especially challenging in the climactic battle sequence when the ships are temporarily lost in cloud during a storm and the ground is not visible as a visual reference. As the sequence progressed and the cloud cover opened to show more land, we could prepare viewers for shots later on over Paris where the airships are much closer to the ground.”

Airship Design
The airship design was extremely complex and originated from three sources. Art Department concepts were the first step, from which scale-model maquettes were made, at the same time as partial sets were under construction of the decks for the live action shoot. This sometimes gave the team slightly conflicting references to work with, because a set extension based on the maquette, for example, might not accurately match the set build. They had to walk a fine line between these two references.
Furthermore, a fair amount of detail was not covered by any of the three sources. The scale model wasn’t able to represent the 80m airship in fine enough detail, and interior details were often not represented anywhere, so Mr X had to pitch some interior designs to the director based on looks in the concepts. They also had to invent some of the mechanicals such as the operation of the sails, rigging or rudder. Because adding detail contributed to the sense of scale of the airships, they modelled individual cannonballs, boxes, coils of rope, tables, barrels and rivets. In one scene when the Duke of Buckingham’s airship lands outside the Louvre, the camera moves so close to the rudder that individual scratches are revealed.

Scaling in Stereo
Working in stereo made conveying scale more complex. The convergence and interocular distance on their Maya cameras had to be set to make the airships feel large whether they were close to the viewer, 100m away or up in the sky. Their experience from a previous project - shooting stereo footage from a helicopter using RED cameras on a rig with a 6in interocular distance – helped them understand how much stereo is in a scene and build an effective CG camera set up accordingly. Nevertheless, a compromise always exists between realism and drama for this kind of work.

Showing ripples of wind affecting the fabric of the sails and the ships balloons further helped convey scale. A cloth team built tools into the balloons so the animators could adjust how much effect the wind would have on the sails and balloons whether they were in stormy or calm skies, or landing and stopping.

The environment, weather and lighting was always important aspects of storytelling in this film, for a few reasons. The clouds afforded attractive compositions and expressed dramatic cues for different levels of tension or humour in a scene. The Musketeers use the cloud as a place to hide during the stormy battle. The matte painting department developed several concept paintings for the storm and from these, the digital FX team projected layers of clouds onto geometry.

Dramatic Skies
Using layers to take advantage of the 3D not only made the shots more dynamic but allowed them to produce a more convincing stereo effect. They tried to place objects in the foreground whenever possible. The aerial camera view, as it looked at the ships in the mid-ground, included wisps of cloud passing very close to camera as the foreground, with the background surrounding all of this. This set up produced the immersive quality in the 3D, drawing the audience into the picture, especially effective for CG green screen shots and set extensions.

Scene by scene, the skies would change from dark and overcast, to blown-out to beaming with sunlight. The airships’ journey to Paris needs to reach a climax as they pass through the storm and the team enhanced the drama through the lighting effects in the sky. When D’Artagnan triumphs at the end, the sky glows at the dawn of a new day with god rays shining down from the sun.
Consequently, a custom cloud system was developed for the project, starting with a large matte painted hemisphere as the background layer. The darker part of this sky indicated south, the light part was north, in line with the story. Over this background and mid-ground, clouds rendered in Houdini from the effects department were added as required.

At the heart of the storm these clouds and mist would move in around the airships, using a more complex simulation to allow foreground mist to curve around the CG objects. Meanwhile the team had to ensure everything was correctly layered to work in stereo. When the airships flew into a cloud bank, all elements had to be lined up perfectly to hold the correct illusion of disappearing into the clouds.

Photo Survey
The build of Notre Dame Cathedral started with sending a team to Paris for a photo survey, supplemented with historical documents. A matching Notre Dame cathedral is also located in Rheims, where another team was sent to complete a photo survey with RED cameras for an accurate sense of scale. They had to take some license with the design, partly because the set build wasn’t an exact replica of the roofline, and certain camera angles had to be adjusted to create a suitable back drop.
When one of the airships collides with the cathedral, a dramatic dynamic simulation had to be set up for the roof tiles breaking off and the damage to the roofline. To prepare the asset for this, all affected interiors had to be completed and the structure made, in effect, ‘water tight’. While it was modelled in Maya, the effects simulations were done in Houdini, so the models had to work properly across both applications, which required attention to all details and continuous checking for compatibility down the pipeline.

For example, if a CG cannon shot a ball into an airship, the wood needed to splinter and fly off following realistic physical behaviour from all shooting angles. These simulations were computationally expensive and often tied up the render farm as they worked out the best process.

Rank and File
The film’s version of the Louvre was based on a shoot at an historic Bavarian location, Würzburg Palace. One of the camera’s approaches to it was a long cable cam shot using a cable that passed over the building starting at ground level, rising over a statue up to a balcony where the king and queen were standing. The avenue leading up to it and the forecourt was filled with thousands of digital soldiers in rows, standing at attention.

“Fifty live action soldiers were included at the shoot but 24,000 more were added digitally using Mr X’s own crowd system and animation cycles, resulting in ranks of living, breathing soldiers. Each was made to subtly shift weight or slightly sway, while the ranks were interspersed with isolated actions to add further signs of life.

The Palace building standing in as the Louvre was a photographic element, but the grounds around it were CG, replacing the location’s current surroundings of a modern city. The element was isolated and a forest was added behind with the Seine River along one side to form the aerial view. The gravel forecourt where the soldiers stand is now a car park that the team made about five times larger. The environment was created from textured assets and projected or conventional matte paintings, always varied according to the proximity of the camera and the need to address parallax.

Duelling Doubles
The fight between D’Artagnan and Rochefort on a precarious rooftop at Notre Dame required a hybrid approach to CG and digital double replacement. The shot of D’Artagnan swinging into view, Tarzan-style, onto the roofline was full CG with both actors as digital doubles. “Almost all wide shots used doubles, combined with a few plate elements. Others needed almost everything replaced - except the actors. Chunks of the set build, such as the peak of the roof, could be included for them to stand on. A different solution was required almost anytime the camera angle changed.

“Another fight at the Tower of London features an explosion bursting through the Tower wall to the exterior. A flaming Beefeater falling from a burning window is actually a photo element while the hero D'Artagnan, leaping to safety onto the airship above them is replaced digitally. The Tower itself is also CG, and the small portion of the airship was in the plate and extended.

“When we researched the Tower for this shot, we did the usual book research and photo survey on foot, but also surveyed from a helicopter to give us reference at the same aerial camera angles the director wanted in the CG shots. An action movie needs shots with dynamism,” Eric said.

Global Illumination Pipeline
This was one if the first movies Mr X has worked on using a full global illumination pipeline. “Earlier, GI was too expensive on the render farm to be practical”, Trey Harrell explained. “But since adopting V-Ray as the primary lighting renderer, the GI is virtually free. It adds a pleasing aesthetic to the shots and was used on almost all the shots for ‘The three Musketeers’.

“Traditionally in VFX, particularly with Renderman style pipelines such as those based on PRMan, Mantra or 3Dlight, artists would light with spotlights and try to mimic light bouncing around the room, or sunlight or fill light from the sky bouncing up off the ground. But now with physically based lighting tools artists could add for example, a Kino Flo 4 Bank to one side of a scene and have it render as if that lamp had been on set. They can achieve multiple light bounces intuitively without resorting to traditional ‘cheats’.”
Using GI also meant the digital doubles, airships and other assets could be surfaced and textured and calibrated to behave consistently under any lighting set up. Eric had spent a week in Germany on set to carry out texture scanning of the actors for digital doubles. They built a digital version of this scanning stage, taking all of their texture and photo references for doubles and props, recreating the process in Maya and V-Ray, including the exact same lighting set up used during photography, to be able to make objective comparisons with the reference photography. The approval process could thus literally be looking at the real photos right next to the CG, sometimes head to waist.

Aerial Views
Because of the airships, ‘The Three Musketeers’ involved aerial views and approaches to various locations, like the Louvre and the Tower, and the team had to assess several factors when deciding how much photo imagery should be included to enhance the photorealism and how much to rely on CG. It depended on how close to camera they worked, how much detail was needed, and how much parallax was in the shot. CG trees were a particular challenge when trying to introduce enough natural looking motion into them to prevent a static look without whipping them into a storm. Too subtle motion at a distance often looked like noise.

During the aerial battles, some of the full CG shots showed cannons firing into the airships, fire erupting on impact, smoke flying past the sails and dissipating. Each hit ended up being custom-made. They had intended to build up a library of cannon fire effects but the individual camera angles in one shot meant creating hero simulations for each one.

Compositing was done in Nuke, in which the stereo tools were especially useful for subtly changing the stereo position of objects in a scene to improve the balance and focus attention. For example, when the airship crew begins throwing gear overboard to lighten it and the objects fall into camera on the ground, the director wanted to give the shot more impact. They were able to use Nuke tools to give more offset to objects to make them fall further into the audience. These tools also helped correct anomalies in the plate photography such as locational offset and keystoning.


Magical Realism
Given the level of influence the environment had in this story, Trey’s role as Lighting Supervisor was influential. He regarded the fine line the film took between fantasy and reality as a challenge to achieve ‘magical realism’. It’s not serious but rarely strays into comedy. To get started on lighting design , the team looked at reference from the ‘Pirates of the Caribbean’ films and ‘Stardust’ to ‘Master and Commander’ and some of Paul’s previous films.

The light was also vital for the VFX as they involved so many CG assets. Lighting for realism is one thing but they also needed to communicate scale and mood and to give the audience clues for tension, optimism and humour. The best approach was to start each lighting scenario with the onset light data as usual and they proceed based on emotional cues. Few sequences ended up sticking strictly to HDR references.

The amount of texturing and detail on the doubles, airships and Paris itself demanded an upgrade to their entire surfacing and texturing pipeline. Mr X was one of the first effects houses to purchase licences of Mari from The Foundry, which was adopted at the very beginning of the project. Without it, Trey doesn’t think they could have completed all the work, with as few staff as they had, in time.

Balloon Textures
An example of the volume of texturing is the balloons on the airships. “One of these balloons contains 64 4K texture tiles with very high texture resolution, because in some shots the camera is just inches from the surface. All digital doubles and the hulls of the airships again represented very high texture resolution due to camera proximity combined with the demands of stereo – scratches and bolts all had to be visible.

Many approaches were applied to lighting a given scene, to set up interactive lighting for cannon blasts, for example, the ships might be rendered in V-Ray, explosions and smoke in Houdini while the debris was simulated in Houdini but rendered in V-Ray so the y would cast accurate shadows and bounce lighting back into the shot and mesh with everything else. At other times, the traditional methods worked, like throwing in an orange spotlight to light up an explosion in the side of a ship.
When lightning storms hit, they might render out lightning flash passes but they also now have a re-lighting pipeline. Trey said, “The compositing department has begun using 2.5D and true 3D compositing a lot more in the last few film projects. In one situation on ‘Three Musketeers’, compositing artists could interactively add lightning flashes from arbitrary directions, for example on top of Notre Dame, to match with set lighting and CG lighting as well.

Hybrid Escape
On another occasion, a giant airship passing over the Louvre was handled with projections from shadow passes rendered out of their daylight system – or they might find traditional roto mattes worked best. All such decisions were made on a per shot basis, choosing the most efficient approach that would make sense and work for the show.”
As D’Artagnan flees the bombs thrown into the Tower of London to escape on the stolen airship, the team took on a very hybrid approach to the whole shot, as Eric also described above. Trey said, “Practical photography included the Beefeater on fire, filmed elements of explosions projected back onto explosion cloud geometry, both matte painted and full CG elements of the Tower plus a postage stamp of the interior set inside the Tower, deliberately exposed for visibility. In addition a digital double of D’Artagnan leaps across to the almost entirely CG airship.”

In one layer, interactive light from the explosion affecting the double and the side of the ship was handled both as a V-Ray render pass and another Houdini render pass for fill light, then the re-lighting pipeline was used, where the compositors can place lights and flashes into a scene for detail, plus extensive moving roto mask work.

Twice the Time
Trey talked about timing in a stereo 3D project like ‘The Three Musketeers’ on which their teams were working from the very start. “Despite, their frequency these days, 3D movies can still easily take twice as much time in post production as conventional 2D films, especially from a rotoscoping and compositing perspective. Stereo rotoscoping required on true stereo productions is specialised and painstaking. From a lighting point of view, it can likewise demand twice the intensity of effort - much of this due to render farm demands.

“On ‘The Three Musketeers’, asset artists and animators were onto the shots from previs to help design them and once they were approved and shot, the rotoscoping department started immediately on clean up for hair and mattes, pulls keys, and makes decisions on where to seam up live action with CG material.” Then models are surfaced and textured and built while the asset and lighting teams work out look development to balance the final shot, and may have to pitch some looks to the VFX supervisor and director.

Getting to the stage when effects animation is approved might take three months, after which tasks tend to split between the effects and lighting teams. After several more weeks of work, shots are presented in dailies and revised until they are right. “Planning definitely pays off even on shots that seem easy because you always end up having to think on your feet on the ones you least expect,” Trey said.

Details