科研项目

项目列表


Event Detection and Recognition from Videos


视频事件检测与识别


Introduction

 

Deep Spatia-temporal Neural Networks for Multimedia Event Detection

 

In this project, we propose a novel method using deep spatial-temporal neural networks based on deep Convolutional Neural Network (CNN) for multimedia event detection. To sufficiently take advantage of the motion and appearance information of events from videos, our networks contain two branches: a temporal neural network and a spatial neural network. The temporal neural network captures motion information by Recurrent Neural Networks with the mutation of gated recurrent unit. The spatial neural network catches object information by using the deep CNN, to encode the CNN features as a bag of semantics with more discriminative representations.

 

Figure

 

 

Papers

 

1. Jingyi Hou, Xinxiao Wu, Feiwu Yu and Yunde Jia. Multimedia event detection vis deep spatial-temporal neural networks. [PDF]

 

 

Recognizing key segments of videos for video annotation by learning from Web Image Sets

 

In this project, we propose an approach of inferring the labels of unlabeled consumer videos and at the same time recognizing the key segments of the videos by learning from Web image sets for video annotation. The key segments of the videos are automatically recognized by transferring the knowledge learned from related Web image sets to the videos. We introduce an adaptive latent structural SVM method to adapt the pre-learned classifiers using Web image sets to an optimal target classifier, where the locations of the key segments are modeled as latent variables because the ground-truth of key segments are not available.

 

Figure

 

 

Papers

1. Hao Song, Xinxiao Wu, Wei Liang and Yunde Jia. Recognizing key segments of videos for video annotation by learning from Web Image Sets. Multimedia Tools and Applications, 2016. [PDF]