Skip to content

Latest commit

 

History

History
73 lines (52 loc) · 4.27 KB

introduction.md

File metadata and controls

73 lines (52 loc) · 4.27 KB
Raw

Development of Human Activity Recognition Component of Robocomp

Objectives

  • Classify human activities with >95 % precision
  • Extend classification to different datasets
  • Extend classification to different environments

Future development

  • Extend to several humans in one video
  • Follow up of a person, i.e. give same unique id to a person

Outline of tasks to follow

First month (work on CAD-60):

  1. Read and open dataset
  2. Divide datasets to environments
  3. Feature extraction.
  4. Test a few classifiers. Start with a multi-class linear SVM as mentioned in these papers.
  5. Improve classifiers. Change DBMM base classifiers to my proposals.

Ideas for second month:

  1. Join environments from CAD-60
  2. Work now on other datasets.

Ideas for third month:

  1. Improving code?
  2. Tunning parameters, concluding project.

Datasets

Dataset Name Link # subjects # activities # joints # environments # total samples
CAD-60 link 4 12 15 5 60
CAD-120 link 4 10 (+10 subactivities) 15 --- 120
UTKinect-Action link 10 10 20 --- ---
Florence 3D-Action link 10 9 --- --- 215
MSR Action3D link 10 20 20 567
MSR DailyActivity3D link --- 20 --- --- 320
SYSU link 40 12 20 --- 480
UWA 3D Multiview II link 10 30 20 --- 1076
SBU Kinect link 7 8 15 --- 300
KARD link 10 18 --- --- ---

Articles

A Probalistic Approach for Human Everyday Activities Recognition using Body Motion from RGB-D Images, Diego R. Faria, Cristiano Premebida, Urbano Nunes. In IEEE RO-MAN'14: IEEE International Symposium on Robot and Human Interactive Communication, 2014.link

A Human Activity Recognition System Using Skeleton Data from RGBD Sensors, Enea Cippitelli, Samuele Gasparrini, Ennio Gambi, and Susanna Spinsante. In Computational Intelligence and Neuroscience, 2016. link

3D human action segmentation and recognition using pose kinetic energy, Junjie Shan, Srinivas Akella. IEEE International Workshop on Advanced Robotics and its Social Impacts, 2014.

Human Action Recognition by Representing 3D Skeletons as Points in a Lie Group, Raviteja Vemulapalli, Felipe Arrate and Rama Chellappa. CVPR 2014 link

A Human Activity Recognition System Based on Dynamic Clustering of Skeleton Data, Alessandro Manzi, Paolo Dario and Filippo Cavallo, 2017 link

Explanation of scripts

  • classes.py -- code where class Person() is defined, with other necessary functions that are needed for class Person (read the data activities from folders)

  • read_data.py is where we create objects of class Person() and use its functions

  • To add: separation into environments. Feature extraction

Questions

  1. What is the extra file on the CAD-60 folders about
  2. Details about on-fly-testing. Is it done on the same CAD-60 datasets? Has fly-testing used together with reinforcement learning?
  3. How many layers does the ANN base classifier has in DBMM? (It has 40 neurons in hidden layers with hyperbolic tangent sigmoid activation function and normalized exponential (softmax)
  4. What else would be done in second and third months?
  5. How is pose distance defined?