Product Review Analysis

This learning projects aims to analyze the sentiment of users on twitter about a particular brand or product. This is done by developing a pipeline using Apache Airflow which extracts tweets, perform sentiment analysis using open source LLM from Hugging Face and store the results in PostgreSQL. The results are then visualized using a dashboard.

Setup

1. Install dependencies: Install the dependencies using the conda environment file environment.yml
2. Authentication secrets: Make a secret.json file in the root directory of project with the following structure:

{
    "huggingface": {
        "token": "HUGGING FACE API TOKEN"
    },
    "postgresql": {
        "host": "HOST",
        "dbname": "DATABASE NAME",
        "tablename": "TABLE NAME",
        "user": "USER",
        "password": "PASSWORD"
    }
}

3. Database: Create a database in PostgreSQL with the name DATABASE NAME and a table with the name TABLE NAME with the following structure
4. Run the airflow webserver and scheduler by make run_airflow
5. Run the dashboard by make run_dashboard

Screenshots