Live-Dash

Stream processing pipeline for analyzing live chat data from Twitch IRC

Providing actionable insights to live streamers

• Live video channels receive thousands of messages every minute and it is difficult to read and make sense of these messages

• This project aims to create a dashboard to help streamers identify insights
• This can help streamers in identifying the audience engagement

Architecture

Live-Dash runs on the AWS cloud, using the following cluster configurations:

• 1 t2.large RDS PostgreSQL instance
• 3 m4.large EC2 instances for Kafka brokers and Kafka producers
• 3 m4.large EC2 instances for Spark
• 1 t2.medium Web-Server

Setup

• I used pegasus to spin up my clusters. You might want to read the following to setup the enviornment on your localhost to test this out.

UI (Still there is lots of room for improvement!)