Hello world! This is the project’s summary that describes the project, plain and simple, limited to the space available:
A comprehensive analysis of the factors influencing animal intakes and outcomes in shelters and centers distributed among five regions across the United States.
These factors include animal gender, type, and breed as well as socio-economic indicators like population, per capita income and unemployment rate.
PROJECT PHILOSOPHY • PROTOTYPING • TECH STACKS • IMPLEMENTATION • HOW TO RUN?

A python based ETL project that would extract, transform data from different web sources and load them into a tabular database (PostgreSQL) in an attempt to provide a full scope analytical understanding of animals journeys in US shelters.

User Types

1. Data Engineers.
2. Data Analysts.
3. Pet Adopters and Foster Families.
4. Regional Animal Welfare Organizations.
5. Veterinarians.

User Stories

1. As a Data Engineer:
   
   • I want to automatically scrape various economic indicators from reputable sources so that our dataset is always up-to-date.
     
   • I want to integrate different data sources seamlessly. Ensure fault tolerance in our data pipelines, so that potential failures don't interrupt our analyses.
2. As an Analyst:
   
   • I want to query the database.
     
   • I want to visualize the data using Power BI.
3. As a Pet Adopter or a Foster Family:
   
   • I want to access a dataset that provides information on specific animals availability, behaviors, and characteristics to ensure proper care.
     
   • I want to know about available animals for fostering and temporary care.
4. As a Regional Animal Welfare Organization:
   
   • I want to oversee and coordinate animal adoption processes and welfare efforts at a regional level.
     
   • I want to assess the overall welfare of animals in the region and to allocate resources effectively.
5. As a Veterinarian:
   
   • I want to understand trends in animal deaths and euthanasia rates.
     
   • I want to coordinate with shelters or rescue organizations.

We have designed our projects to webscrape, through an ETL project and including it in a PowerBI Sample Dashboard,

Logger File

app.log

Data Flow Diagrams

Schema Diagram

Animal Intakes and Outcomes Analysis is created using the following technologies:

Frontend

Interactive PowerBI Dashboard: A central dashboard where viewers can view:

1. Animal Characteristics and Socio-economic Indicators:
   Graphs, charts and visualizations displaying animal traits and key socio-economic metrics over time.
2. Interactive filters:
   Options to filter data by date, region, or specific animal features for customized views.

Backend

1. API Integration and Web scraping.
2. ETL Pipeline: using Python and Pandas, raw data is extracted, transformed into a usable format and loaded into PostgreSQL database.
3. Database: Schema Design - Indexing - Data Integrity - Backup & Recovery.

Report Screenshots & GIFs

Home Page

Overview

Adoption Analysis

Socio-economic Analysis

Summary Analysis

Available for Adoption

Toolboxes

In Overview Page	In Adoption Analysis Page	In Socio-economic Analysis Page

Report Recap Video

powerbi.mp4

To set up Animal Intakes and Outcomes Analysis follow these steps:

Prerequisites

Hardware & Software:

• A computer/server with sufficient RAM and processing power.
• Operating system: Linux (preferred for production) or Windows.
• Required software: Python (3.x), PostgreSQL, Git (for version control).

Dependencies:

• Install the necessary Python libraries: pandas, requests, numpy, BeautifulSoup, psycopg2, openai.
• Install database connectors/drivers for PostgreSQL.

Setting Up the Environment:

Clone the Repository:

git clone https://github.com/ola-smaha/Animal_Intakes_and_Outcomes_Analysis.git

Set Up the Database:

• Start the PostgreSQL server.
• Create a new database and user with the appropriate permissions.
• Run any initialization scripts to set up tables or initial.

Running the Backend:

Start the Data Ingestion & ETL Process: python etl_script.py

You should be able to check the app.log file to see the ETL work.

As for the dashboard access: Please use this link to access your data.