Vizrt Looks Ahead at Live Broadcast Production
Digital Media World talked to RV Krishnan, Vizrt Vice President of Graphics for the APAC region and General Manager of Vizrt Australia Richard Hewitt about some of the biggest media innovations that will change live production over the next several years, including automation, cloud production, IP distribution, augmented reality and artificial intelligence.
We wanted to find out how Vizrt is using these innovations in products for news, sports and entertainment production.
Storytelling will always be the most important part of journalism. What is evolving is how journalists tell their news stories and where audiences experience them. Richard Hewitt and RV Krishnan talked about cloud-based story techniques at Vizrt that have the potential to shift newsroom production toward workflows driven by journalists who need to reach audiences on all platforms.
Journalist-driven workflows give journalists a chance to be more creative as well as work faster, and take over the delivery of complete stories from any location, using cloud systems to store and access content, and software that can search, find and re-purpose content for different destinations.
Vizrt's tools already give journalists ways to use data visually – for instance, they use Vizrt's election graphics software to control live visualisations of the statistics they are reporting on in various ways. But the idea of creating once and distributing across many platforms is powerful for a journalist. They can now re-purpose existing video content in custom resolutions and formats, add Vizrt 3D graphics and publish their stories online to social media, VOD platforms or websites.
Viz Story is a relatively new application that puts looks and presentation as well as content under the control of the journalist, instead of relying on a dedicated graphics team. It can also adapt finished content to different devices and destinations. Viz Story suits online breaking news because it only takes a few minutes to make news stories available for multi-platform publishing. It uses templates, an approach Vizrt uses in several of its applications, as a quick way for journalists to add Text On Video (TOV) content for social media sites, while controlling the look and style. All added graphics are then rendered in real-time.
RV Krishnan said, “Cloud systems are very useful in this context for making tools and data available and as a vehicle for delivering stories, although it's important to remember that the cloud is still in a transition stage in terms of how it can be used. Much more development in the cloud will continue to unfold.”
He also commented on the open platform behind all of Vizrt's products and their compatibility with HTML. “Vizrt's software was originally designed this way to make sure it could be integrated with existing newsroom computer automation systems. Now, HTML has become the basis for web production, stories and graphics,” he said. “All tools have the same foundation and exist on the one platform. You could run the entire Vizrt platform through the browser for field editing if required.
Viz One asset management software, for example, has an embedded HTML video player that supports web-based production, with frame accurate editing functions. Viz Story was also built on this open platform, which will allow the progressive evolution of its software components. Vizrt products are designed to work with content management systems, CDNs and legacy and future online publishing tools.
“Vizrt are software developers, before any of the other specialities the company now has,” RV Krishnan said. “All products are based on the Viz Engine real-time graphics rendering engine, compositing platform and video server, Viz One media asset management system and Viz Artist, a real-time 3D graphics, modelling and animation tool and broadcast production designer.”
Richard commented that the fact that, since April 2018, Viz Artist itself is a free tool, gives scope for a wider range of users – journalists in particular – to learn how to use it to suit their applications. It is the core software Vizrt artists use to build basic geometry and create fonts for their lower third graphics, construct virtual sets and design custom 3D animations.
Its functionality includes global illumination, native support for Substance materials, Cinema 4D LiveLink, and files from all the major 3D modelling applications, and it also supports using video inside the creation process – that is, videos can be used as part of a scene, or as textures mapped on geometry via UV coordinates. Viz Artist is able to integrate clips of any resolution or live stream video as part of the scene design.
Openness comes through Viz Artist's scripting capabilities for scene logic, and the ability to write custom plugins to solve a specific challenge in a workflow. As part of the Vizrt integrated workflow, designers design scenes in Viz Artist and make them accessible as templates in other Vizrt applications for journalists and production staff creating online stories.
Flexibility and Agility in Broadcast
Flexibility and agility in broadcast are not new themes but are continuing to gain importance, partly because standards are still in progress. RV Krishnan said that standards are, and will be, as critical as ever, but he also sees a lot of caution in the industry regarding standards. Although IP distribution looks set to become, over time, the primary means of broadcasting video, for now broadcasters need to build infrastructures that IP stream video and graphics in any format, while producing a SDI signal for on-air use at the same time.
IP-based Viz Engine composites graphics and video in formats such as MPEG-2 at, for example, 1080i/25, 1080i/29.97, 1080p/30, 720p/50 and others for online, mobile and live on-air applications. At distribution time, video and graphics are all composited live for the screen size and resolution needed for each platform, which is not only a faster approach but also opens opportunities for custom branding and sponsorship across platforms without extra render time.
Vizrt's tools use IP streaming and automatic device detection and adaptation in order to distribute content anywhere. Viz One can deliver content both as linear playout and to multiple platforms. Because all audiences have mobile devices today, graphics need to be displayed in a way that is easily understandable on every device whether full-screen or composited as part of a video. Vizrt assigns an appropriate scale and design for the graphics without altering the message – that is, optimising the end user experience.
As well as its interoperability, Viz Engine is also one of the most flexible pieces of software used in broadcast today, illustrated by its use of modern graphics hardware, and ability to support SD, HD, 4K/UHD media and cloud rendering. Viz Engine renders 3D scenes in real-time, producing animations at HD, 4K and higher resolutions, and can be deployed in a cloud or other virtualised environment, in an IP facility or from a data centre.
Thus, through its ability to take advantage of competitive, high-performance graphics hardware, producers can use Viz Engine as the graphics layer for applications that need interactive, high-fidelity graphic content. Viz Engine will also run in the IBM cloud, so that editors can use Vizrt’s graphics plug-in in their native NLE to access graphics without dedicated Viz Engine hardware. The combination of the plug-in API and an ASCII communication protocol creates a foundation for various applications.
Viz Engine's open structure has made possible Vizrt's progress in Augmented Reality, one of the more eye-catching parts of their presentations this year at IBC where Vizrt partnered with tracking company Ncam Technologies and with the Unreal Engine. The three companies have been developing AR tools for eSports, and the ability to combine template-based graphics and texts with Unreal Engine game assets and graphic environments. The result is realistic looking graphics supporting storytelling within live productions.
On the exhibit stand at IBC, a motion capture company supplied real-time retargeted skeleton data directly into the Viz Engine as a live stream, which allowed an interactive AR 3D model to interact with the presenter. To control the looks of these new capabilities, new bone and skin-based mesh deformation tools are combined with physically based rendering. HDR functionality in Viz Engine supports HLG and PQ HDR formats, and also compatibility with the Sony S-Log3 HDR format for digital intermediate processing.
Looking at broadcast as a business, automation represents a chance for companies to adjust to the fact that the dynamics of the business are changing. The traditional established companies see new threats like OTT and internet delivery cutting into their viewership. They feel pressured to deliver the same or more content to multiple platforms while relying on the same staff. RV Krishnan said, “Automation is here to stay. It is cost-driven. Businesses are experiencing revenue stress and see efficiency as the best way forward. They need to manage with the minimum of errors, do more with less and improve the return on investment on every product. If they have a traditional facility they are asking what can be automated, and how much.”
Richard said, “The proliferation of content formats continues to drive interest in automation. At one end are 4K cameras and other new sources and at the other, the need to repurpose media depending on the destination. Video processing involves repeating tasks over many times when outputting to different formats, which is prone to error and labour-intensive.”
However Vizrt's newsroom software Viz Opus and Viz Mosart are further examples of using templates to automate live programme rundowns. Viz Mosart takes input from the newsroom computer system and translates it into commands for connected systems and devices in the studio, while producers keep control over last-minute updates, systems integration and real-time processing. The broadcaster creates reusable automation templates so that one newsroom script can be aired anywhere, from any studio at any time.
Broadcasters become more agile, respond to events more rapidly and save on costs. In breaking news situations, Viz Mosart operators can make live changes on the fly using the system’s shortcuts and rundown modification tools. Because one central operator can create templates used at more than one studio, operators can come from a wider pool of users, not only technicians or people working as a team.
Regarding AI, Richrd's and RV Krishnan's responses were more cautious. Richard said, “In these early days especially, it's important to decide where it makes sense, where it adds value. For example, right now it can add value to content management systems and speed up the automation of tasks such as subtitling, but it needs to be driven by a business-oriented purpose.”
RV Krishnan also said, “AI is moving forward gradually. For now, it relies largely on metadata. Vizrt has no specific AI products so far, but remaining an open platform will help incorporate external engines, for example, into existing products.
“One example of software that does use machine learning to a certain extent is Viz Eclipse, which inserts targeted and region-specific ads into television feeds, and potentially generates new revenue streams for rights holders through localised advertising. Eclipse replaces existing field-side advertising boards with virtual signage in real time. It doesn’t alter the boards themselves or disrupt TV production workflows because its camera tracking and keying are image-based – no information has to be collected from the location.” www.vizrt.com