Second Workshop on Mathematical Analysis of Images in Bordeaux

Registration Venue Accomodation

Miguel Granados: Two methods for automatic video inpainting.

Project Page

Video inpainting methods are often required in industries such as movie and TV production. For instance, when unwanted objects like pedestrians or cars appear in the scene, or when crew members that are required for the shot need to be removed in post-processing. However, there are no robust video inpainting methods that can handle arbitrary motions of objects and camera, let alone be applicable to high resolution videos used in the industry. Additionally, in the interaction of our research group with the movie industry, we learned that the removal of unwanted objects from videos is still performed by artists in a frame-by-frame basis. To address the general video inpainting problem, I take advantage of the redundancy found in video sequences to restore the scene behind the unwanted objects: Objects are removed by reusing other views of them that are available in other video frames. We proposed two automatic video inpainting methods that follow this principle: One for removing of objects that occlude other dynamic objects from videos taken with static cameras, and one for removing objects that occlude other static objects from videos taken with dynamic cameras. The inpaint result is produced by enforcing local consistency constraints using a global optimizer (graph cuts). In the experimental evaluation, we found that high quality inpaintings can be produced whenever there is enough redundancy in the video, even for high resolution videos. Our hope is that, as automatic methods for video inpainting become faster and more reliable, this type of editing tasks can be made less time consuming for the users, and finally become widely available to the general public.