initial

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2018 huailiang
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,131 @@
+# Video to Pose3D
+
+> Predict 3d human pose from video
+
+<p align="center"><img src="asset/kunkun_alphapose.gif" width="100%" alt="" /></p>
+
+## Prerequisite
+
+1. Environment
+   - Linux system
+   - Python > 3.6 distribution
+2. Dependencies
+   - **Packages**
+      - Pytorch > 1.0.0
+      - [torchsample](https://github.com/MVIG-SJTU/AlphaPose/issues/71#issuecomment-398616495)
+      - [ffmpeg](https://ffmpeg.org/download.html)
+      - tqdm
+      - pillow
+      - scipy
+      - pandas
+      - h5py
+      - visdom
+      - nibabel
+      - opencv-python (install with pip)
+      - matplotlib
+   - **2D Joint detectors**
+     - Alphapose (Recommended)
+       - Download **duc_se.pth** from ([Google Drive](https://drive.google.com/open?id=1OPORTWB2cwd5YTVBX-NE8fsauZJWsrtW) | [Baidu pan](https://pan.baidu.com/s/15jbRNKuslzm5wRSgUVytrA)),
+         place to `./joints_detectors/Alphapose/models/sppe`
+       - Download **yolov3-spp.weights** from ([Google Drive](https://drive.google.com/open?id=1D47msNOOiJKvPOXlnpyzdKA3k6E97NTC) | [Baidu pan](https://pan.baidu.com/s/1Zb2REEIk8tcahDa8KacPNA)),
+         place to `./joints_detectors/Alphapose/models/yolo`
+     - HR-Net (Bad 3d joints performance in my testing environment)
+       - Download **pose_hrnet*** from [Google Drive](https://drive.google.com/drive/folders/1nzM_OBV9LbAEA7HClC0chEyf_7ECDXYA) | [Baidu pan](https://pan.baidu.com/s/1v6dov-TyPVOtejTNF1HXrA)), 
+         place to `./joints_detectors/hrnet/models/pytorch/pose_coco/`
+       - Download **yolov3.weights** from [here](https://pjreddie.com/media/files/yolov3.weights),
+         place to `./joints_detectors/hrnet/lib/detector/yolo`
+     - OpenPose (Not tested, PR to README.md is highly appreciated )
+   - **3D Joint detectors**
+      - Download **pretrained_h36m_detectron_coco.bin** from [here](https://dl.fbaipublicfiles.com/video-pose-3d/pretrained_h36m_detectron_coco.bin),
+        place it into `./checkpoint` folder
+   - **2D Pose trackers (Optional)**
+      - PoseFlow (Recommended)
+        No extra dependences
+      - LightTrack (Bad 2d tracking performance in my testing environment)
+        - See [original README](https://github.com/Guanghan/lighttrack), and perform same *get started step* on `./pose_trackers/lighttrack`
+
+
+
+## Usage
+
+0. place your video into `./outputs` folder. (I've prepared a test video).
+
+##### Single person video
+
+1. change the `video_path` in the `./videopose.py`
+2. Run it! You will find the rendered output video in the `./outputs` folder.
+
+##### Multiple person video (Not implemented yet)
+
+1. For developing, check `./videopose_multi_person`
+
+   ```python
+   video = 'kobe.mp4'
+
+   handle_video(f'outputs/{video}') 
+   # Run AlphaPose, save the result into ./outputs/alpha_pose_kobe
+
+   track(video)					 
+   # Taking the result from above as the input of PoseTrack, output poseflow-results.json # into the same directory of above. 
+   # The visualization result is save in ./outputs/alpha_pose_kobe/poseflow-vis
+
+   # TODO: Need more action:
+   #  1. "Improve the accuracy of tracking algorithm" or "Doing specific post processing 
+   #     after getting the track result".
+   #  2. Choosing person(remove the other 2d points for each frame)
+   ```
+
+
+
+
+##### Tips
+0. The [PyCharm](https://www.jetbrains.com/pycharm/) is recommended since it is the IDE I'm using during development.
+1. If you're using PyCharm, mark `./joints_detectors/Alphapose`, `./joints_detectors/hrnet` and `./pose_trackers` as source root.
+2. If your're trying to run in command line, add these paths mentioned above to the sys.path at the head of `./videopose.py`
+
+## Advanced
+
+As this script is based on the [VedioPose3D](https://github.com/facebookresearch/VideoPose3D) provided by Facebook, and automated in the following way:
+
+```python
+args = parse_args()
+
+args.detector_2d = 'alpha_pose'
+dir_name = os.path.dirname(video_path)
+basename = os.path.basename(video_path)
+video_name = basename[:basename.rfind('.')]
+args.viz_video = video_path
+args.viz_output = f'{dir_name}/{args.detector_2d}_{video_name}.gif'
+
+args.evaluate = 'pretrained_h36m_detectron_coco.bin'
+
+with Timer(video_path):
+    main(args)
+```
+
+The meaning of arguments can be found [here](https://github.com/facebookresearch/VideoPose3D/blob/master/DOCUMENTATION.md), you can customize it conveniently by changing the `args` in `./videopose.py`.
+
+
+
+## Acknowledgement
+
+The 2D pose to 3D pose and visualization part is from [VideoPose3D](https://github.com/facebookresearch/VideoPose3D).
+
+Some of the "In the wild" script is adapted from the other [fork](https://github.com/tobiascz/VideoPose3D).
+
+The project structure and `./videopose.py` running script is adapted from [this repo](https://github.com/lxy5513/videopose)
+
+
+
+## Coming soon
+
+The other feature will be added to improve accuracy in the future:
+
+- [x] Human completeness check.
+- [x] Object Tracking to the first complete human covering largest area.
+- [x] Change 2D pose estimation method such as [AlphaPose](https://github.com/MVIG-SJTU/AlphaPose).
+- [x] Test HR-Net as 2d joints detector.
+- [x] Test LightTrack as pose tracker.
+- [ ] Multi-person video(complex) support.
+- [ ] Data augmentation to solve "high-speed with low-rate" problem: [SLOW-MO](https://github.com/avinashpaliwal/Super-SloMo).
+

python/.gitignore

@@ -0,0 +1,196 @@
+# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio and WebStorm
+# Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839
+
+# User-specific stuff
+.idea/**/workspace.xml
+.idea/**/tasks.xml
+.idea/**/usage.statistics.xml
+.idea/**/dictionaries
+.idea/**/shelf
+
+# Generated files
+.idea/**/contentModel.xml
+
+# Sensitive or high-churn files
+.idea/**/dataSources/
+.idea/**/dataSources.ids
+.idea/**/dataSources.local.xml
+.idea/**/sqlDataSources.xml
+.idea/**/dynamic.xml
+.idea/**/uiDesigner.xml
+.idea/**/dbnavigator.xml
+
+# Gradle
+.idea/**/gradle.xml
+.idea/**/libraries
+
+# Secret
+.idea/**/deployment.xml
+.idea/**/remote-mappings.xml
+
+# Gradle and Maven with auto-import
+# When using Gradle or Maven with auto-import, you should exclude module files,
+# since they will be recreated, and may cause churn.  Uncomment if using
+# auto-import.
+# .idea/modules.xml
+# .idea/*.iml
+# .idea/modules
+# *.iml
+# *.ipr
+
+# CMake
+cmake-build-*/
+
+# Mongo Explorer plugin
+.idea/**/mongoSettings.xml
+
+# File-based project format
+*.iws
+
+# IntelliJ
+out/
+
+# mpeltonen/sbt-idea plugin
+.idea_modules/
+
+# JIRA plugin
+atlassian-ide-plugin.xml
+
+# Cursive Clojure plugin
+.idea/replstate.xml
+
+# Crashlytics plugin (for Android Studio and IntelliJ)
+com_crashlytics_export_strings.xml
+crashlytics.properties
+crashlytics-build.properties
+fabric.properties
+
+# Editor-based Rest Client
+.idea/httpRequests
+
+# Android studio 3.1+ serialized cache file
+.idea/caches/build_file_checksums.ser
+
+checkpoint/*
+outputs/*
+data/data_3d_h36m.npz
+data/own2DFiles/*
+
+
+
+__pycache__/
+*.py[cod]
+
+
+*.pyc
+.ipynb_checkpoints/
+*.gif
+*.jpg
+*.png
+*.npz
+*.zip
+*.json
+*.mp4
+*.tar
+*.pth
+*.weights
+*.avi
+*.caffemodel
+*.npy
+
+
+/st_gcn
+/outputs
+/nohub*
+/VideoSave
+/ActionRecognition   
+/work_dir
+
+
+
+
+
+
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+
+eggs/
+.eggs/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+.hypothesis/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# pyenv
+.python-version
+
+# celery beat schedule file
+celerybeat-schedule
+
+# SageMath parsed files
+*.sage.py
+
+# dotenv
+.env
+
+# virtualenv
+.venv
+venv/
+ENV/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+
+# Self-defined files
+/local_test/
+/lab_processing/