JARVIS - Your Personal AI Assistant

Welcome to the JARVIS project! This web-based application is designed to act as a personal AI assistant and is capable of voice commands, facial expression analysis, and more. This README provides an overview of the code structure, features, and application usage.

Overview

This project is a simple web-based AI assistant called JARVIS. It features a user interface where you can interact with JARVIS through voice commands, capture video from your webcam, and process face detection using the face-api.js library.
This application consists of two main parts:

1. HTML and CSS - Defines the structure and styling of the web page.
2. JavaScript - Handles the functionality for voice recognition, facial expression analysis, and interactive features.

Features

• Interactive User Interface: Includes a microphone button to initiate voice commands, a video feed, and buttons to capture and process video frames.
• Face Detection: Uses face-api.js to detect, and analyze faces in the video feed.
• Audio Feedback: JARVIS provides spoken responses to user commands.

Code overview

1. HTML and CSS components:
   • Meta Tags: Ensure proper character encoding, compatibility, and responsive design.
   • CSS Links: Includes external stylesheets for Font Awesome icons and Google Fonts.
   • Internal Styles: Styles for layout, colors, fonts, and responsiveness.
   • Audio Elements: For playing introductory and activation sounds.
   • Main Section contains:
     • Input button: To start voice recognition.
     • A video container for capturing video and a canvas for facial recognition analysis.
     • Buttons: Capture, Process, and Change My Mood


2. Javascript Component
   The Javascript file adds interactivity and functionality to the application.
   • Imports: Loads the face-api.js library for facial expression analysis.
   • Model Loading: Loads pre-trained models for face detection, landmark recognition, and expression analysis.
   • Event Listeners:
     • Capture Button: Accesses the camera and streams video to the video element.
     • Process Button: Analyzes facial expressions from the video feed and updates the canvas.
     • Change My Mood Button: Performs sentiment analysis based on facial expressions and responds accordingly.
   • Voice Recognition:
     • SpeechRecognition: Uses the Web Speech API for voice command recognition.
     • Command Handling: Processes voice commands to perform various actions like opening websites or providing information.
   • Facial Expression Analysis:
     • Analyze Expressions: Calculates the average expression values from detected faces and responds based on the predominant emotion.
     • Based on the detected sentiment, JARVIS can open relevant content or provide comforting messages.
     • Based on the detected sentiment, Jarvis can plot the expression-timeInterval graph.

How to use

1. Opening the application: Load the HTML file in your Chrome browser as Speech recognition works only on Chrome*
2. Interacting with JARVIS:
   • Click the microphone button to start voice recognition
   • Use voice commands like "Jarvis, Open Wikipedia", "Jarvis, What's the time?" and "Jarvis, Play Believer".
     Remember to use "Jarvis" before any command you give!
     

     !! Also, make sure to give your command within 4 seconds after Jarvis starts listening. !!

   • Click the Capture button to start the video stream. This also starts the data collection for plotting the sentiment vs Time interval chart.
   • Click the Process button to analyze facial expressions
   • Click the Change My Mood button to analyze facial expressions and get a response based on detected emotions.
   • To display the plot: Run the last block of code to get the plot of sentiment vs time-interval.

-* Our tests include Brave and Chrome

List of Commands available

Remember to use "Jarvis" before any command you give!

1. Hello: Replies Hello
2. What is your name? : Gives its name
3. Open Google: opens google.com
4. Introduce yourself: Gives an intro
5. Open YouTube: Opens youtube.com
6. Open Facebook: Opens facebook.com
7. Search "keyword": Searches for the "keyword" on google.com
8. Chatgpt: Opens chatgpt.com
9. Bitcoin: Opens Bitcoin chart
10. Game: Opens game center
11. Wikipedia "keyword": Searches for "keyword" on Wikipedia
12. Time: Gives the current time
13. Play "keyword": Plays the "keyword" on Youtube
14. Date: Gives today's date
15. Shutdown: Stops the voice recognition
16. Many more commands that will be searched on Google.

Dependencies

1. face-api.js: For facial expression analysis.
2. Google Fonts: For the custom font "Roboto Mono".

Conclusion

This README provides an overview of the JARVIS web application, including its structure, features, and code functionality. Feel free to explore the code to fit your needs. For any issues, please do contact us.

Collaborators List

1. G Madhava Asrith Murthy
2. Panshul Jindal
3. Shreyansh