Halpe Full-Body Human Keypoints and HOI-Det dataset

What is Halpe?

Halpe is a joint project under AlphaPose and HAKE. It aims at pushing Human Understanding to the extreme. We provide detailed annotation of human keypoints, together with the human-object interaction trplets from HICO-DET. For each person, we annotate 136 keypoints in total, including head,face,body,hand and foot. Below we provide some samples of Halpe dataset.

Download

Train annotations [Baidu | Google ]

Val annotations [Baidu | Google ]

Train images from HICO-DET

Val images from COCO

Realtime Demo with tracking

Trained model coming soon! Stay tuned with AlphaPose

Keypoints format

We annotate 136 keypoints in total:

    //26 body keypoints
    {0,  "Nose"},
    {1,  "LEye"},
    {2,  "REye"},
    {3,  "LEar"},
    {4,  "REar"},
    {5,  "LShoulder"},
    {6,  "RShoulder"},
    {7,  "LElbow"},
    {8,  "RElbow"},
    {9,  "LWrist"},
    {10, "RWrist"},
    {11, "LHip"},
    {12, "RHip"},
    {13, "LKnee"},
    {14, "Rknee"},
    {15, "LAnkle"},
    {16, "RAnkle"},
    {17,  "Head"},
    {18,  "Neck"},
    {19,  "Hip"},
    {20, "LBigToe"},
    {21, "RBigToe"},
    {22, "LSmallToe"},
    {23, "RSmallToe"},
    {24, "LHeel"},
    {25, "RHeel"},
    //face
    {26-93, 68 Face Keypoints}
    //left hand
    {94-114, 21 Left Hand Keypoints}
    //right hand
    {115-135, 21 Right Hand Keypoints}

Illustration:

26 body keypoints

68 face keypoints

21 hand keypoints

Usage

The annotation is in the same format as COCO dataset. For usage, a good start is to check out the vis.py. We also provide related APIs.

Evaluation

We adopt the same evaluation metrics as COCO dataset.

Related resources

A concurrent work COCO-WholeBody also annotate the full body keypoints. And HOI-DET for COCO dataset is also available at V-COCO

Citation

The paper introducing this project is coming soon. If the data helps your research, please cite the following papers at present:

@inproceedings{fang2017rmpe,
  title={{RMPE}: Regional Multi-person Pose Estimation},
  author={Fang, Hao-Shu and Xie, Shuqin and Tai, Yu-Wing and Lu, Cewu},
  booktitle={ICCV},
  year={2017}
}
@inproceedings{li2020pastanet,
  title={PaStaNet: Toward Human Activity Knowledge Engine},
  author={Li, Yong-Lu and Xu, Liang and Liu, Xinpeng and Huang, Xijie and Xu, Yue and Wang, Shiyi and Fang, Hao-Shu and Ma, Ze and Chen, Mingyang and Lu, Cewu},
  booktitle={CVPR},
  year={2020}
}