Junaid Baig, who heads the Applied Technology team at Dimension, talks about using open source software to improve AI pipelines and set up customised, studio-driven workflows.
Dimension is a virtual production company with studios in London and Rome, as well as mobile volumetric video capabilities it delivers to clients’ sets around the world. Finding that AI and open-source tools are attracting attention among the film and TV producers they work with, Dimension can see the industry shifting away from traditional off-the-shelf (OTS) solutions as productions look for ways to cut costs and speed up project timelines.
At the same time, Dimension has been developing their own open source AI pipeline tools. Digital Media World had a chance to speak to Junaid Baig, Dimension’s Head of Applied Technology, whose team is using open source applications to improve AI pipelines and set up customised, studio-driven workflows.
Combined Skills
“The mix of skills we have at Dimension are uniquely ours. Our Applied Tech team are the ones responsible for the technological innovations and developments that sit under the Dimension Futures banner. Their expertise includes specific engineering skills that are instrumental to taking advantage of open source tools and tweaking them to be more useful for us as a business,” Junaid said.
“However, the skills and background of our technical artists are critical as well. Looking at a pipeline from an entirely technical point of view isn’t always helpful – you need people with an understanding of art and creative production to look at the tools we’re using and give insight into how an artist can get the best out of them. That’s the case whether it’s open source, AI-driven, or something off-the-shelf.”
A Two-Team Approach
As a result, Dimension works with two teams. One is a team of generative AI (GenAI) engineers focussed on creating and fine-tuning machine learning models, and the other is the team of artists who work under the production to create the final content. “Their collaboration is such that the GenAI Engineering team is always evolving the tools and integrating them into the production pipeline for the artists, and meanwhile, the artists always have the latest and greatest of our GenAI models,” said Junaid.
Dimension has recognised that relying exclusively on OTS software can create certain workflow challenges. A workflow can become siloed at certain points, especially if singular pieces of software deliver just a small part of a pipeline. That will be where the use for each tool starts and stops, limiting the value of investing in it.
Junaid said, “This issue isn’t necessarily overcome by integrating AI or open source tools. It’s more about looking at the pipeline as a whole and making sure the component parts – whatever they are – are the right ones for the job.
“However, the ability to create our own version of a tool does mean that it’s engineered to work how we need it to. That saves a huge amount of time and resources because we know the production process will lead to the exact outcome that we need without having to go through multiple programmes or software applications. The control that using open source software gives us is what we can use to really help save time on projects.
Unreal Engine – Re-engineered
“A good example of how this works is DUE – our own version of Unreal Engine. It’s been engineered to deliver on-set benefits that the standard Unreal Engine doesn’t give us. As a result, not only does that save us the time on set of having to set up other programmes to deliver what we need if we’re shooting with an LED volume. It also saves the time and money otherwise spent on post-production, because the images we’re capturing in-camera are exactly what the filmmaker wants to see.”
Dimension doesn’t think about open source tools project-by-project. Rather than the project, it’s more often a question of what the demand is. If a piece of software or a tool already exists that they can use to get the right outcome, then certainly they will use it. But if there isn’t something already in place, they might take an open source tool and create something themselves that will help achieve a specific result.
Getting artists to think and work in new ways involves training, though not in the usual sense. Every artist employed by Dimension has, of course, already been trained on sophisticated 3D DCC (Digital Content Creation) software like Unreal Engine, Maya, Houdini, where they have learned with experience to control every aspect of scene building.
“But with our implementation of GenAI to produce videos from crafted images, we essentially bypass most of the scene building process that artists are accustomed to, and this evolved way of constructing scenes has been a formidable experience for the artists," Junaid said. "This approach focuses on creating the look and feel instead of handling complex tools. We have seen most of the artists embracing the fact it is their own art they are creating without the use of complex tools and methodologies. But the new tools still have a long way to go to be useful for every artist in every day work.”
AI as a Tool
Dimension’s emphasis on using the most appropriate tools available for pipelines also applies to artificial intelligence. Their approach to AI is the same as the approach to any other technology-based challenge. “What tool is going to give us the best possible result?” said Junaid. “The answer to that question will be what we’ll use.”
They use open-weight models as building blocks for their pipeline, and then finetune these models on their dataset created and crafted by the artists. These tools allow them to create their own working pipelines and connect them to existing production pipelines and tracking systems. From there, these new pipelines allow faster iteration without the overhead of a complex 3D process in the early stages of the production.
Beyond Visualisation – Faster Iteration
Many of the benefits of using AI are about speeding up the iterative process, rather than helping artists to better visualise their ideas. Junaid said, “We still need artists to bring their ideas to life – they’re just able to use AI tools to do it faster than before, including getting faster feedback to get to the next stage of iterations quicker.”
Junaid, the Applied Technology team and a separate team of just two artists recently undertook a special project exploring the possibility of an end-to-end GenAI first pipeline. All those involved have learned a lot from this project, as it gave them specific goals to work toward.
He said, “The intention was to tie into Dimension’s Script to Screen proposition that we use with clients. We went about creating a demo piece that could be handled entirely by two people. We were aware that, like many projects that use AI now, it was going to be the fine-tuning of the models and creating the workflows that integrate with our existing production pipeline, that would take the longest time.
“However, every every single aspect was massively accelerated once the team grasped the ‘new’ way of working with the models. The creative aspects in particular meant the artists could focus more on creating the art than wrangling tools. Challenges that remained were certain inconsistencies that, due to self-imposed time constraints, we couldn’t resolve in time. While those inconsistencies bothered the artists at the time, the GenAI Engineering team took that as input and resolved those issues for subsequent productions.” dimensionstudio.co
