netflix has released a video object removal model called EMPTYdeveloped by researchers from Netflix and Sofia University. The model is available on Hugging Face for public use and is not limited to Netflix productions.
VOID means Object removal and video interactions. Unlike typical video painting tools that remove objects and fill the space with static background content, VOID also predicts how the remaining objects in a scene would physically behave once the removed element disappears.
What the VOID video object removal model does
Given a video of two vehicles colliding, VOID can remove one vehicle and generate images showing the remaining vehicle continuing down the road, with post-impact debris, smoke, and fire replaced by an intact road surface.
In another example from the article, removing a person jumping into a pool results in a video in which the surface of the pool appears intact, with no splashes in the water or nearby ground. The model takes as input both the video and a language description of the object to be removed, making it a vision and language system rather than purely visual.
How VOID compares to other tools and where you can use it
Netflix researchers compared VOID to several existing tools, including Runway, Generative Omnimatte, DiffuEraser, ROSE, MiniMax-Remover, and ProPainter. In a survey with 25 participants in multiple scenarios, VOID was preferred 64.8 percent of the time, with Runway in second place at 18.4 percent.
The authors mention in the preprint that “VOID excels at modeling the complex dynamics that can follow object removal.” These are the researchers’ own descriptions and the article has not yet been peer-reviewed.
VOID is available on Hugging Face. Netflix has not announced any plans to incorporate the model into existing products or projects. The authors of the preprint article are Saman Motamed, William Harvey, Benjamin Klein, Zhuoning Yuan and Ta-Ying Cheng of Netflix, along with Luc Van Gool of Sofia University.
