Nvidia announces an AI-driven green screen for streamers

Bottom line: With TwitchCon only a day away, Nvidia has announced the RTX Broadcast Engine. The new RTX-accelerated SDKs utilize the RTX GPU’s “dedicated AI processors called Tensor Cores” to add various effects to live streams.

The most exciting and useful of these for the serious streamer is RTX Greenscreen. Chroma key compositing (colloquially Green Screen) is the method of filming a subject on a green backdrop and then substituting the background with another image. This is how many streamers insert themselves into other content like gameplay.

However, using chroma keying can be a hassle. The background screen takes up a lot of studio space. The lighting must be just right, and the streamer has to avoid wearing anything green for the effect to work.

Nvidia notes, RTX Greenscreen uses artificial intelligence to identify the part of the image that is human and then remove the background. It works in real-time and appears to be just as effective as the traditional method. Sufficient lighting is probably still essential.

A second SDK called RTX AR detects the user’s face. It can track facial features and expressions, even creating a model on which other 3D effects can be applied and changed in real-time. One application would be to use your face to control an animated model much the same way Apple’s Memojis work.

The third application is RTX Style Filters. These filters “use an AI technique called style transfer to transform the look and feel of a webcam feed based on the style of another image.” Streamers can use this to give their feed a sketched look or other strange-looking effects.

Keep in mind that right now, these features are only available in the RTX Broadcast Engine SDKs. However, Nvidia mentioned that is has been working with OBS, a popular live-streaming application developer, on integrating RTX Greenscreen into OBS Studio. It will demo it at TwitchCon this weekend, and should roll out to users in about a month.

The company said Streamlabs and XSplit are also working on adding the effects to their apps. It had no word on when those companies would be ready with updates.

By: Techspot