Extracting Features from Videos

This directory contains the code to extract features from video datasets using mainstream vision models such as Slowfast, i3d, c3d, CLIP, etc. The only requirement for you is to provide a list of videos that you would like to extract features from in your input directory.

If you have any question with the code, please contact [email protected] for further support.

Structure of the Repository

• models/: This is the directory where you customize the models (e.g. structure, preprocessing, cropping) you want to use for feature extraction.

• configs/: This directory contains configuration files that define different runtime settings

• condor/: This directory contains the code to submit computation to CHTC for throughput computing, ignore if not applicable.

Installation

• Follow INSTALL.md for installing necessary dependencies.

Instructions

• Pass in the path to the config file.

• Make sure to modify the fields in the config file if necessary

python main.py --config configs/egovlp.yaml 

Note: For egovlp, i3d, and c3d, you are REQUIRED to manually download pretrained checkpoints . Please refer to models/README for more details.

Customization

• To custoimze preprocessing (transformation, cropping, and mirroring), please modify the implementation of the get_transform method under models/model_name.py

• To use customized model or your own checkpoints, please modify the implementation of the load_model method under models/model_name.py

Adding a new Model

1. Add a new yaml file to configs/. Check out configs/README for more instructions.

2. Add a new python file to models/. Check out models/README for more instructions.

Acknowledgement

This directory is built on top of the Ego4d directory .

Appendix

To resize the video or to convert the video into a different frame rate, refer to this repository .