Pixomondo’s Phil Prates, Johny Berbsov and Asad Manzoor take an inside look at the environments, characters, FX and interactive previs they created for ‘Halo’ Season 1.
In the Paramount+ streaming series ‘Halo’ Season 1, fans of the video franchise find themselves in a familiar world of Halo with characters they know, but involved with a new story. The characters engage in a war taking place in the 26th Century between the United Nations Space Command and the Covenant, an allied force of aliens colluding to destroy the human race.
Pixomondo’s VFX team completed over 500 shots across the series, led by VFX Supervisor Phil Prates. Their scope of work was wide and varied, encompassing many of the Reach City and Hangar environment shots, and the Raas Plateau environment. They were responsible for all shots of the series’ digital character Cortana, and also created the well known Covenant aliens, the Brutes, Grunts, Jackals and Atriox, and their weapons.
From Start to Finish with Halo
Sound knowledge of the Halo game canons and an ability to confidently use the game as source material were critical. The artists downloaded hours of game references, gameplay videos and Halo cinematics and used them for very specific details on top of client notes and art directions. Phil said, “We always started from the reference of the games when designing any of the characters, weapons and vehicles. We wanted to make sure all the parts of the Halo look and feel were converted from that known, identifiable starting point.”
Pixomondo was involved in the project as early as May 2019, and began with the design of characters, props, vehicles and environments, then moved into early season shot work. Covid interrupted the filming, pushing later episode shot work all the way out until early 2022. never the less,, the slowdown gave them precious time to work on details and build better concepts.
About the shoot, Phil said, “It’s fair to say the production for Halo uses a mix of everything – massive set builds with green screen for extensions, full green screen shoots with only the ground remaining in-shot, lots of filming on location, plus dedicated shoots used strictly for motion capture. However, support from the production’s on-set team was excellent. Their staff supplied us with all survey data, set detail and measurement data, so we didn’t need to visit the sets ourselves.”
Interactive Previs – Unreal Engine
Previs was a critical stage when blocking events ahead of the shoot, and also also served as a guide in post-production. The production had a dedicated team working on previs, mainly focusing on blocking out large scale events with complex camera moves, such as the battles. Pixomondo picked up from there, continuing the development of much of this work, and also created techvis to plan how to shoot the events on location.
Through the team’s ability to work in Unreal Engine, the production was able to be interactively involved with the previs stage, controlling all action. It was an interesting process on its own. Pixomondo’s Head of Virtual Content Asad Manzoor said, “The initial pitch to use Unreal for previs began in the early stages of pre-production. We were tasked with sequences, specifically the chief's fights in episode 5 and 9.
“We were essentially taking assets from the traditional departments at this time – asset layout and mocap animation executed in Maya – conforming them to Unreal's standards, and then crafting a playthrough of the sequence as a whole. With the resulting diorama, the director could pick up the project and, by controlling the camera, 'shoot' the sequence while it played out on a monitor.”
Clearer Vision, Happier Accidents
For complex, high action set pieces like Master Chief's big fights, traditional pre-vis simply didn't contain enough visual information. The director and DP needed a clear vision from the very start to execute them smoothly, especially on a television timeline.
They would have to block in their shots quickly and by eye, see what their lighting would look like on the day (episode 5 was shot outdoors), and consider where lighting may need to be supplemented — all combined with the visual effects elements that would be seen in the final frame including explosions, Covenant gore, bullet impacts and so on.
“Shifting the pre-vis to Unreal meant everyone could not only SEE a faithful representation of their final frame incredibly early, but also feel the timing of all of the smaller bells and whistles in context,” said Asad.
“For instance, the Episode 5 fight was a sequence consisting of motion capture data Pixomondo had recorded in-house and bashed together with various assets from the Unreal Engine Marketplace, and edited via Unreal's Sequencer. Being able to add FX and lighting on-the-fly meant happy accidents were much more frequent – in a sense, this made the process feel much closer to an actual shoot than a traditional pre-vis back-and-forth.”
Virtual Benefits
Once it had been decided that pre-vis would be going through the virtual production department, Pixomondo received detailed scans of the sets and drone footage so they could recreate the outdoor locations as digital elevation models and use them for previs and layout for episode 5. Then they started set-dressing and modifying the scene in-engine, to prepare for the final build.
Using Unreal allowed them to interact with the production in interesting ways. CG Supervisor/Head of CG Evgeny (Johny) Berbasov said, “For the quarry battle, we worked very closely with Jonathan Liebesman who was directing the episode. He is very familiar with Unreal techniques and wanted to design the sequence interactively with our previs, animation and Unreal team.
“Once the main action was blocked, each artist was responsible for producing a very large chunk of animation that hooked up to the next artist's work. Every few days, we would deliver an Unreal scene to Jonathan with all the segments stitched together and he would spend time in Unreal designing cameras and cutting the sequence. As we approached a final edit, the Unreal team would then devote time to sweetening the lighting and adding FX.”
Reach City
Pixomondo’s environmental work included Reach City, the Reach hangers and runways, Dr Catherine Halsey’s apartment and the Raas mountain top. “Our reference came mostly from real world photography,” said Phil. “Typically, we would source our designs from the games and the art department, as well as have a portion of the environment that would be shot in the plate. We sourced our own real world elements and photographs that matched the look and feel as a guide, while still anchoring our work to the look of the practical portions of the sets.”
The impressive cityscapes, huge and complex, started with a kit bash base for the buildings, adjusting and augmenting the designs as needed. “Scale was the key to making this approach work. By designing the assets within the city to scale, from the city blocks all the way down to cars on the roads and boats in the water, it was possible to move the camera from super wide shots down to tight looks at specific areas,” Phil said.
Procedural and Modular
Johny and the CG team designed every environment with modularity in mind. Components could be swapped from background to hero resolution based on the camera proximity. He said, “The Reach City environment had major hero buildings directed by client concepts and the rest of the building blocks were generated procedurally in Houdini. We created multiple parts and levels of the stylized buildings.
“Based on a city map containing roads, rivers and parks, our system created unique building blocks in which none of the buildings repeated themselves, looking very organic all together. We also introduced procedural road traffic, boats in the rivers and spaceships patrolling the city. Lighting the environment needed a different approach, though, because we had to maintain city light conditions throughout the day and night, all with moving components.”
The Reach Hangar environments were central to the story, as well as very complex. They required the team to build shots that the action returns to throughout the season – from the opening episode when Master Chief goes rogue, all the way until the end when Halsey’s ship crashes on the tarmac.
“The Hangars were treated as repeatable assets, as they ran all around the Reach City’s main tower,” said Johny. “It was impossible to manage all levels at once, so we used a modular system again and installed them as far as the eye could see in the shot. Main level hangars and VIP level hangar were pretty much the same but with different scales and unique set dressings. Lighting was a huge part of the hangar looks and it wasn’t trivial to make huge interiors look good. The lighting team made every hangar unique with dedicated light setups, in some cases consisting of up to a thousand lights.”
Favourite Bad Guys
Pixomondo was responsible for several different alien species from Covenant for the Halo series – Grunts, Jackals, Brutes and Atriox, all of which are very familiar to game fans. Although the characters are all based on their essential looks from the games, they are completely re-designed with a new look for this show.
This task represented a challenge but also an opportunity to make them more realistic and engaging in the shots.
Phil said, “Having the game as our source material meant the resolution on the texture maps was quite forgiving. While making the characters, we focussed on realistic damage, and wear and tear on the armour and weapons. These characters are warriors and this story was unlikely to be their first time in battle. Also, the colour pallete from the game is very vibrant and uses a lot of very saturated colours. We maintained the main tones while dialing back their vibrant nature to bring them closer to a real-world paint job.
The game also gave them a backstory to base their behaviour and animation on, but they wanted to take control and introduce a few of their own ideas as well. For instance, the Jackals have very snappy, jerky movements and behaviour. “We were using bird and reptilian characteristics to make them take on this distinctive look. Atriox is a character that is also Brute but has a unique armour and weapon design. As a leader to the Brutes, he needed to look similar but have his own characteristics that set him apart,” said Phil.
“We got to do a fantastic sleeping Grunt shot as well. Throughout the games, you come across these little guys napping away and so we were very excited about bringing that kind of shot to the fans.”
In the Shoes of Master Chief
Across the series, Pixomondo used both keyframing and motion capture, alternately or in combination, to animate their characters, depending on what was happening in the shots and the style of motion required. For example, they held a full motion capture day to record battle action for the final Raas battle, after blocking out the main actions in a rough animation pass.
“In order to use real motion as a guide, we captured specific motion with stunt performers to get the pacing and subtle body movements. Then we took that back into our scenes and expanded on it. This gave us a great base to start from before keyframing all the nuanced behaviours and actions. We also shot our own in-house motion capture for the same reasons, providing a base pass to start from,” Phil said.
“All the action was designed to look and feel like the games. The POV shots, in particular, were designed to give the audience a front row view of what it would be like to be in the shoes of Master Chief. We would go for very frantic camera moves and close combat shots, making the viewer feel immersed in the action. The helmet edges would frame the view, and the visor and graphics would be added to support the scene. The mini maps and ammo counters were animated to match the events happening all around the viewer.”
Associated with the characters are numerous vehicles, including Warthogs, Pelicans and Condors, that Pixomondo made sure would feel real to the audience, as well as look interesting. He said, “We took each asset from its recognised game look to a final quality vehicle that felt tactile and authentic. We also designed the cannons and structures on real military weaponry designs. The animation of these massive, scaled objects then needed to be tuned to sell their enormous size and weight.”
Re-Inventing Cortana
Cortana is a special character, different to all the others. She is played by Jenn Taylor, the same actress that takes her role in the Halo video games, and her facial and body performances are created through motion capture. Johny said, “Cortana was our main human character. We built her from the start with photorealistic looks in mind, and later on she became a holographic asset.
“We ran a lot of tests and went through many variations before coming to the final look you see in the show. She has a complex structure with more than 1,000 joints and roughly 800 blendshapes for the face. We also made a library of supporting animation clips, on top of the actress’ facial and body motion captured from the set.”
Phil remarked that transitioning Cortana from the game was actually easier than expected. “With Jenn Taylor’s voice and performance for the facial and body capture, she was bringing the Cortana attitude to every scene,” he said. “Our job was to take this base and apply it to the digital Cortana asset. Her shape and form were adapted from a collection of game references that went through a long series of concept stages before finalising the look and feel of the character that people know and love.
“In some cases, Jenn was simply on set in the motion capture suit and facial rig. These shots were about first removing her from the plates, and then introducing the digital Cortana to the same location. This was a great method because the other actors could clearly interact with her, which made for a really present character.”
Scaling and Integration
But there were also a number of shots in which Cortana appears as a small figure on a desk or the dashcontrol of the Condor gunship. Since they couldn’t shrink Jenn down to the necessary size, her performance was shot separately and meanwhile the actors had a miniature 3D printed Cortana model with them on set to look at during the shoot.
The facial capture was mainly done on location. “This data was always used as our performance base and our starting point,” said Phil. “The audio from that capture was used directly in the episodes. Our animation on top was done to enhance the performance, but we always needed to stay true to the original movement. Jenn’s and Cortana’s facial features are actually quite different, so the captures would need to be applied to Cortana, then adjusted to bring the feeling from Jenn’s performance to the Cortana face.”
As a hologram, integrating Cortana into the plates needed special consideration. “Technically, she was was made up of photons (the smallest possible unit of light) and connecting energy, and these elements would break apart and come together to form the character,” Phil said. “The lighting that Cortana would leave in the environment was always played very subtly to indicate that, in the story, she is a highly advanced, futuristic hologram. Furthermore her look adapts to the lighting in the environment she occupies, taking on characteristics that show she is in the same room that we are in.” www.pixomondo.com
