Indeed-Scraper-Analysis

Overview

This program uses the BeautifulSoup library to search through the job search website www.Indeed.com and scrape data regarding job listings on the site. This program will send the scraped data to a MySQL database where it will organize the data accordingly. The program can then visualize the data from the database in various forms of graphs.

Indeed Scraper Analysis

An awesome indeed scraper that will store and analyze the results!
Report Bug · Request Feature

Table of Contents

1. About The Project
   • Built With
2. Getting Started
   • Installation
3. Usage
4. Roadmap
5. Contributing
6. License
7. Contact

About The Project

The project was used for researching the current job listings in my area for the Computer Science field. It aided me in writing a research paper that further analyzed the results gathered from the program.

Here's why I felt it needed to be created:

• It was an interesting project
• Infinitely faster than manually gathering data from the site
• I can more efficiently organize the data and convert into visuals
• Easy to use and friendly towards users who know nothing about the subject

A list of commonly used resources that I find helpful are listed in the acknowledgements.

Built With

I used Python and the BeautifulSoup library to scrape all of the data. Meanwhile, SQL was used to store the data in one location.

• [Python]
• [SQL]
• BeautifulSoup

Getting Started

This is an example of how you may give instructions on setting up your project locally. To get a local copy up and running follow these simple example steps.

Installation

1. Clone the repo

   gh repo clone Alejandro-Vega/Indeed-Scraper-Analysis

2. Set up a MySQL database to store the data, although the program will still run if a database is not provided, but it will not store it anywhere

Usage

Run the program and choose from the various options in the Python console.

Options:

• Scrape the Indeed website for data
• Gather Statistics (Convert database data into graphs)

This programs takes into consideration many different topics such as degrees, interpersonal keywords, intrapersonal keywords, date, remote jobs, salary, and many more.

Contributing

Currently not allowing for contributions from other people. Although, if you find an issue with the program feel free to open an issue request.

License

Distributed under the MIT License. See LICENSE for more information.

Contact

Alejandro Vega - LinkedIn - [email protected]

Project Link: https://github.com/Alejandro-Vega/Indeed-Scraper-Analysis