'Audio Creatures' Make a Splash of Sound, Light and Motion

The annual Vivid Festival in Sydney takes light as both its theme and its medium. The Harbour precinct and other areas of the city glow with illuminated artworks including projections, sculptures and live performance. Vivid opens with the ‘Lighting the Sails’ event on the festival’s first night, when motion graphics are projected onto the sail-like roof of the Sydney Opera House and played out with a vibrant soundtrack. This event brings a special look and feeling to the festival each year that everyone anticipates.

This year, Spinifex Group produced the ‘Lighting the Sails’ event with a project titled ‘Audio Creatures’. The concept and story came from local director Ash Bolland, and the animations themselves were created by motion artists Tim Clapham from his company Luxx studio and Mike Tosetto, director of Never Sit Still, who aligned their work with original music by Brazilian sound designer Amon Tobin. This project won the Bronze Award in the Live Event category at AEAF Awards 2017.

An Audio Creature

The idea for the creatures came from the director, whose concept was to transform the Opera House from a building that is home to music, drama and the performing arts, into live creatures. In other words, beyond just seeing the creatures as projections on the roof, Ash Bolland wanted viewers to regard the Opera House as the creature.

The butterfly in the main image and in the detail [top] and its chrysalis above were shaped according to the lines of the Opera House roof.

To visualize and create the project’s graphics and animation, Ash and Spinifex hired Tim and Mike, whose two companies work as an artists’ collective and share a studio. Audio Creatures involved seven weeks of production, during which the artists produced over eight minutes of content at 4K resolution – a tight schedule considering the duration and resolution of the final renders.

Their creative process on the project stems from a briefing in preproduction with Spinifex Group and the director. Ash showed his ideas for each creature in the story as a collection of style frames, either as Photoshop montages or hand-drawn sketches. The creatures are massive, resembling marine and tidal life and exotic plants, and eventually give way to increasingly mechanical designs by the end of the piece.

Tim said, “At Luxx we have worked on many other projections all over the world, such as a projection in London for Bombay Sapphire. We have some history of working with Spinifex as well, creating projections in Melbourne, Ankara and the USA - including contributing to a world record-breaking projection mapping event that took place in Moscow.”

Experience

Ice Pods.

They worked with Spinifex throughout the process on Audio Creatures, especially to build the transitions between the creatures. To allow the experience to flow as smoothly as possible, they considered the transitions from the very start and developed them over the period of the project. Because Ash was away on a location shoot for a period of the project, we would have production meetings over Skype, and both Luxx and Spinifex would share files via Dropbox. A couple of artists were also working remotely in New Zealand and also in Europe.

As they refined the creatures, Mike and Tim considered both the audience and the director’s ideas. “We knew the audience would be a diverse mix of people and ages and wanted to make sure the look was accessible to as wide a range of people as possible. Ash had designed the creatures to feel friendly, soft and rounded with no sharp or aggressive shapes. Although many of them feel otherworldly, they are engaging and inviting,” said Tim.

“We wanted to the viewers to feel intrigued and fascinated and I think we answered all those demands pretty well by referencing alien-looking creatures from the natural, earthly world, like deep sea animals or complex, intricate insects. We also referenced more familiar examples such as butterflies emerging from chrysalis, and objects breaking open. On the aesthetic side, to pull the whole project together we chose stylised imagery from classic animation such as ‘Akira’, a Japanese animated science fiction film based on a manga story.”

Process

As principle production company, Spinifex supplied Tim and Mike with the technical specifications for the project, plus a 3D model of the Opera House. Their job was to supply the animations as uncompressed movies ready for Spinifex to edit and composite into the complete sequence. From there, Spinifex output the final video splits for The Electric Canvas, the company responsible for the projections onto the building each night.

When the elegant shells crack open to reveal a living octopus, the artists had to show the interior detail at a massive scale.

At the initial briefing, Luxx and Never Sit Still each took responsibility for specific shots, and then proceeded to work on these independently, aligning all renders to the specifications Spinifex had supplied. Their ideas, research and the renders they output were shared with all involved.

Tim and Mike mainly used Cinema 4D to create the motion graphics. “We are very familiar with the software. It allows us to be flexible and fast, which was essential in a project of this scale,” Tim said. They also used X-Particles a fair amount, a particle simulations plug-in that doesn’t require the artist to know Xpresso, Cinema 4D’s scripting tool, or use any other programming. Its dynamics system allows particles to flow, collide and burn.

“The majority of the shots we rendered in V-Ray, which is a fast renderer that gives fantastic results in Cinema 4D,” he said. “We didn’t use V-Ray exclusively, though, but also used Cinema 4D’s own renderer for shots whenever it was appropriate. Our compositor was Adobe After Effects, which also works very well in combination with Cinema 4D.

Natural Numbers

Much of the software’s functionality was used, in fact, including MoGraph, Hair, Cloth, Rigging and a number of expressions were written to help facilitate and automate processes. But a lot of the work came down to hand crafting the shots and animation. Cinema 4D has many ways to solve problems, and often more than one way to achieve the same result.

The images of the Magnetic Core creature above and below illustrate the artists' Python Effector created in Cinema 4D to distribute clones using the Fibonacci sequence.

“For the magnetic core creature, for example, which was much more geometric than our other, softer looks, we tried many techniques and ended up building a custom solution,” said Tim. “This allowed us to create a Python Effector for our scene that would distribute the MoGraph clones using the Fibonacci sequence. The result was a naturally appealing distribution that a random approach could not match. Without this level of control, we would have struggled to find the ideal solution.”

The Fibonacci numbering system appears everywhere in the natural world, such as the leaf and petal arrangements in plants and flowers, which means its numbers apply to the growth of all living things, including a single cell, a seed, a bee hive and also people. Knowing about this sequence isn’t necessary – plants and creatures naturally grow in the most efficient way possible, and subsequently reveal those numbers in growth patterns and arrangements that feel natural and appropriate to us.

Sleight of Hand

The scale and unique shape of the Opera House posed equally unusual animation challenges. As the brief was to transform the building into the various creatures, for the illusion to work, no elements could be allowed to break the edge of the roof. This meant they often couldn’t move the creature in the most direct, natural way, but instead had to constrain the movement within the bounds of the roof.

Consequently many of the creatures were designed with this requirement in mind from the start – they were animating butterfly wings and chrysalis that were shaped to the roof, for instance. At the same time, they still needed to feel natural and organic, rather than forced. Tearing the chrysalis was particularly challenging as the rips had to unfurl inward rather than tear outward, causing them to extend past the roof outline.

Previs and Testing

Throughout the production, the huge size of the projections made it important to keep their eyes on the bigger picture at all times, rather than focussing on minute detail. But working on a computer screen to prepare images for projection onto a massive building was always going to be tricky, due to the substantial difference in scale.

This wide shot of the Jellyfish shows the challenge of constraining the creatures' animation to fit naturally within the bounds of the rooftop.

“For example, we had to create a lot of the animation at a slower pace than usual to compensate for the eventual size of the projection. Furthermore, many factors may be present at outdoor projections that distract the eye from the experience, such as light pollution and local weather patterns. To compete, the colour and contrast in ‘Audio Creatures’ had to be pushed as far as possible, almost severely," said Mike. “What may appear over-saturated on a computer would look much more washed out once it was projected. We wanted to keep the creatures as detailed as we could, while also maintaining a balance with big bold sweeps of colour to ensure legibility and impact.”

For all of these reasons, it is highly desirable to test the projections before the main event, if at all possible. But even so, during the production, rendering out the shots in-house and then compositing them onto an image of the Opera House was really the only way to visualise the results.

CREDITS

Clients: Destination NSW & Sydney Opera House

Sydney Opera House: Brooke Webb, Ben Marshall
Destination NSW: Adam Lowe, Julie Turpie

Interrogate Director: Ash Bolland
Interrogate Executive Producer: Tara Riddell
Audio: Amon Tobin
Sound FX: Marcus Longfoot – Full Circle Audio

Spinifex Managing Director: Cyril de Baecque
Spinifex Creative Director: Jason French

Luxx Creative Director: Tim Clapham
Never Sit Still Creative Director: Mike Tosetto

2D / 3D Artists: Tim Clapham, Mike Tosetto, Dan Braga, Alex Barnet, Nejc Polovsak, Rich Nosworthy, Will Skinner, Duncan Dix, Roy Christian, Pepin Portingale, Nicholas Hunter, Marcus Coblyn.