Machine Applied Data Science for American Sign Language Fingerspelling Recognition
University of Michigan MADS program Capstone Project for Google - American Sign Language Fingerspelling Recognition Kaggle Competition
Feel free to save your own copy of the MADS-ASL-FR_Colab_Walkthrough to your Google Drive and follow through to create your own ASL Transformer model! A Kaggle API key and a Google account are necessary to run your own copy of the Colab Walkthrough - if these are not available, you may view the MADS-ASL-FR Example Completed Walkthrough to view the code for how a framework for a transformer model to translate ASL fingerspelling into text is generated.
The main dataset used for this project is the Google - American Sign Language Fingerspelling Recognition Kaggle Competition dataset, which is publicly available at https://www.kaggle.com/competitions/asl-fingerspelling/data. This dataset is a preprocessed dataset of videos analyzed and transformed by Google’s MediaPipe Holistic model to track the face, each individual hand, and the pose of the body. It is freely available on the competition homepage for download, but discretion is advised since all of the files make up over 180 GB. Accessing the data remotely and temporarily is possible and demonstrated in the MADS-ASL-FR Colab Walkthrough notebook which is linked above.