视频分割--Learning to Segment Instances in Videos with Spatial Propagation Network

Learning to Segment Instances in Videos with Spatial Propagation Network

CVPRW2017

https://github.com/JingchunCheng/Seg-with-SPN

本文针对视频中多目标运动物体分割问题，instance-level object segmentation 提出使用 CNN 网络分三个步骤来解决：前景分割、物体类别识别、分割精细化



3.1. Foreground Segmentation

这里采用类似 FCN 分割框架，采用 ResNet-101 模型，主要的改进：1）去掉全连接层，2）融合多尺度卷积层信息

3.2. Instance-level Recognition

有了前景分割的结果，我们分割出每个物体，further segment instance-level objects，采用的网络和损失函数类似前景分割的网络



4 Mask Refinement

这里我们做了两个步骤的Refinement

1）Spatial Propagation Network 这个网络主要用于学习 affinity entities，The coarse mask is refined under the guidance of the affinity, the learned pairwise relations, for any pairs of pixels.

2）Connected Region-aware Filter 这一步主要 considering temporal information in other frames，the consistency between two frames and helps rectify

some instance confusions



5 Experiments