EraisNET: An Optical Flow based 3D ConvNET for Erasing Obstructions

Abstract

Images captured from behind a fence, window, or during rain generally face occlusions. Though prior works have addressed the problems of individually de-raining, reflection, and occlusion removal, a common approach that removes all the obstruction has found little attention in the literature. In this paper, we address the image occlusion problem by proposing a deep learning-based approach wherein the proposed method uses motion differences between two images and extracts important moving features from videos to separate the background and the obstruction. To accomplish this task, a novel 3D-convolution architecture is introduced, which is trained with synthetically blended videos. We have used learned layer-based CNN methods combined with dense-optical flow with generative networks for better output images. Moreover, a dataset for obstruction removal with sequences for reflection and fencing removal is proposed. The proposed approach is well experimented over a different variety of images and is found as a good candidate against state-of-the-art schemes.