Netflix open-source video erasure model VOID: not only removes objects but also recalculates the physical motion of remaining objects

AIMPACT News, April 14 (UTC+8), Netflix Research Institute and Sofia University INSAIT in Bulgaria jointly developed VOID (Video Object and Interaction Deletion), an AI framework capable of removing objects from videos and re-simulating the remaining scene’s physical behavior. Released under the Apache 2.0 license on Hugging Face on April 3, it is Netflix Research Institute’s first publicly released AI tool.

Traditional video erasure tools excel at filling backgrounds, correcting shadows and reflections, but struggle with scenes where objects are in physical contact (collisions, supports, pushes). VOID’s core ability is understanding physical causality: removing a block in a row of dominoes does not cause subsequent dominoes to fall; removing a person jumping into a pool does not splash water; removing someone holding a guitar allows the guitar to naturally fall.

The technical pipeline consists of three layers:

1. Meta’s SAM2 performs object segmentation, Google’s Gemini analyzes scene semantics, generating a four-value “quadmask” that labels the main object, overlapping areas, affected areas, and background, informing the model not only what to erase but also what will change as a result.

2. The first-stage inference, fine-tuned based on Alibaba’s CogVideoX-Fun-V1.5-5b-InP (a 5 billion parameter diffusion Transformer), generates physically plausible counterfactual trajectories.

3. An optional second stage, “optical flow noise stabilization,” initializes time-correlated noise with the motion predicted in the first stage to prevent object deformation in long segments.

Training data was generated from two sets of physical simulations: approximately 1,900 sets of Kubric rigid body dynamics data and about 4,500 sets of HUMOTO human motion capture data, trained on 8 A100 80GB GPUs. In a preference test with 25 participants, VOID achieved a 64.8% selection rate, significantly outperforming commercial tool Runway at 18.4%. Inference requires over 40GB of VRAM (A100 level). The paper has not yet undergone peer review, and Netflix has not announced plans to incorporate it into production workflows. (Source: GitHub)