Video Segmentation by Non-Local Consensus Voting

Video Segmentation by Non-Local Consensus Voting

Alon Faktor and Michal
Irani

(BMVC 2014)

Paper [PDF]

Abstract

We address the problem of Foreground/Background segmentation of "unconstrained" video. By "unconstrained" we mean that the moving objects and the background scene may be highly non-rigid (e.g., waves in the sea); the camera may undergo a complex motion with
3D parallax; moving objects may suffer from motion blur, large scale and illumination changes, etc. Most existing segmentation methods fail on such unconstrained videos, especially in the presence of highly non-rigid motion and low resolution. We propose a
computationally efficient algorithm which is able to produce accurate results on a large variety of unconstrained videos. This is obtained by casting the video segmentation problem as a voting scheme on the graph of similar ('re-occurring') regions in the
video sequence. We start from crude saliency votes at each pixel, and iteratively correct those votes by 'consensus voting' of re-occurring regions across the video sequence. The power of our consensus voting comes from the non-locality of the region re-occurrence,
both in space and in time - enabling propagation of diverse and rich information across the entire video sequence. Qualitative and quantitative experiments indicate that our approach outperforms current state-of-the-art methods.

Code can be downloaded here

Example Videos: (Press on Video to watch it!)