人群场景的属性--Deeply Learned Attributes for Crowded Scene Understandin

Deeply Learned Attributes for Crowded Scene Understanding CVPR2015

http://www.ee.cuhk.edu.hk/~jshao/WWWCrowdDataset.html

https://github.com/amandajshao/www_deep_crowd

本文要解决的问题是什么了？ 给你一段人群场景的视频，算法能否给出关于这段视频的一些信息？

能否回答下面三个问题？“ Who is in the crowd?”, “Where is the crowd?”, and “Why is crowd here?“

文章总体的流程如下：针对这个问题建立了一个大的数据库，WWW Crowd dataset with 10,000 videos from 8,257 crowded scenes，然后我们对这个数据库人工标记了94个属性，这94个属性是关于上面三个问题 Who Where Why 的 。 接着我们设计了一个 CNN网络 将上面的问题变成一个 CNN分类问题，CNN的输出是 94 类。这里CNN的输入包括两个部分： appearance and motion channels

下面首先来看看我们这个 WWW Crowd dataset 数据库



各个数据库的对比：



94个属性标签主要 分为 三类：

3 types of attributes: (1) Where (e.g. street, temple, and classroom), (2) Who (e.g. star, protester, and skater), and (3) Why (e.g. walk, board, and ceremony).

Crowd Attribute List (94)

indoor, outdoor, bazaar, shopping mall, stock market, airport, platform, (subway)passageway,

ticket counter, street, escalator, stadium, concert, stage, landmark, square, school, beach,

park, rink, church, conference center, classroom, temple, battlefield, runway, restaurant,

customer, passenger, pedestrian, audience, performer, conductor, choir, dancer, model,

photographer, star, speaker, protester, mob, parader, police, soldier, student, teacher,

runner, skater, swimmer, pilgrim, newly-wed couple, queue, stand, sit, kneel, walk, run,

wave, applaud, cheer, ride, swim, skate, dance, photograph, board, wait, buy ticket, check-

in/out, watch performance, performance, band performance, chorus, red-carpet show, fashion

show, war, fight, protest, disaster, parade, carnival, ceremony, speech, graduation,

conference, attend classes, wedding, marathon, picnic, pilgrimage, shopping, stock exchange,

dining, cut the ribbon

人工标记实例：



我们使用的CNN模型



两个网络分支具有相同的结构：Conv(96,7,2)-ReLU-Pool(3,2)-Norm(5)-Conv(256,5,2)-ReLU-Pool(3,2)-Norm(5)-Conv(384,3,1)-ReLU-Conv(384,3,1)-ReLU-

Conv(256,3,1)-ReLU-Pool(3,2)-FC(4096).

最后两个分支合并得到 FC(8192)-FC(94)-Sig producing 94 attribute probability predictions

4.2. Motion Channels



接着分别介绍了 Collectiveness Stability Conflict 的定义和计算

5 Experimental Results

deep learned static features (DLSF)

deeply learned motion features (DLMF)

AUC of each attribute obtained with DLSF+DLMF



Good and bad attribute prediction examples



Compare deeply learned features with baselines



Six attributes predicted by DLSF, DLMF, and DLSF + DLMF