diff --git a/.github/workflows/python-package.yml b/.github/workflows/python-package.yml
index 68e190f..41e5ee2 100644
--- a/.github/workflows/python-package.yml
+++ b/.github/workflows/python-package.yml
@@ -8,6 +8,10 @@ on:
schedule:
# Runs at 12am IST
- cron: '30 18 * * *'
+ pull_request:
+ branches:
+ - develop
+ - master
jobs:
build:
@@ -24,6 +28,11 @@ jobs:
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
+ - name: Set up cache
+ uses: actions/cache@v2
+ with:
+ path: ~/.cache/pypoetry/virtualenvs
+ key: ${{ runner.os }}-poetry-${{ hashFiles('poetry.lock') }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
diff --git a/CHANGELOG.md b/CHANGELOG.md
index f00568e..fa5f25d 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -6,6 +6,14 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+## [3.2.0] - 2021-09-13
+
+### Added
+
+- View run statistics at the end of the script
+- Documentation updates
+- Improved error handling and logging
+
## [3.1.0] - 2021-05-04
### Added
@@ -82,6 +90,8 @@ can continue as normal
project running locally. Suitable for users who are not looking forward to
contribute.
+[3.2.0]:
+ https://github.com/aapatre/Automatic-Udemy-Course-Enroller-GET-PAID-UDEMY-COURSES-for-FREE/releases/tag/v3.2.0
[3.1.0]:
https://github.com/aapatre/Automatic-Udemy-Course-Enroller-GET-PAID-UDEMY-COURSES-for-FREE/releases/tag/v3.1.0
[3.0.0]:
diff --git a/README.md b/README.md
index 00b3f8e..e229b60 100644
--- a/README.md
+++ b/README.md
@@ -1,7 +1,6 @@
[](https://forthebadge.com)
[](https://forthebadge.com)
-# ALPHA IS A PRE DEVELOPMENT BRANCH, DO NOT EXPECT USER FACING ISSUES TO BE ADDRESSED IN THIS BRANCH!
# Udemy Coupon Grabber & Course Enroller: Grab FREE Coupons!
@@ -14,15 +13,16 @@ web-scraping and automation, this script will find the necessary Udemy Coupons
**NOTE: THIS PROJECT WILL NOT WORK WITH NON ENGLISH UDEMY.**
The code scrapes course links and coupons from:
- - [tutorialbar.com](https://tutorialbar.com)
- - [discudemy.com](https://discudemy.com)
- - [coursevania.com](https://coursevania.com)
+
+- [tutorialbar.com](https://tutorialbar.com)
+- [discudemy.com](https://discudemy.com)
+- [coursevania.com](https://coursevania.com)
In case of any bugs or issues, please open an issue in github.
Also, don't forget to **Fork & Star the repository if you like it!**
-***We are also on [GitLab](https://gitlab.com/the-automators/Automatic-Udemy-Course-Enroller-GET-PAID-UDEMY-COURSES-for-FREE)***
+**_We are also on [GitLab](https://gitlab.com/the-automators/Automatic-Udemy-Course-Enroller-GET-PAID-UDEMY-COURSES-for-FREE)_**
**_Video Proof:_**
@@ -32,9 +32,8 @@ Also, don't forget to **Fork & Star the repository if you like it!**
## **_Disclaimer & WARNINGS:_**
-
1. **Use** this ONLY for **Educational Purposes!** By using this code you agree
- that **I'm not responsible for any kind of trouble** caused by the code. **THIS PROJECT IS NOT AFFILIATED WITH UDEMY.**
+ that **I'm not responsible for any kind of trouble** caused by the code. **THIS PROJECT IS NOT AFFILIATED WITH UDEMY.**
2. **Make sure web-scraping is legal in your region.**
3. This is **NOT a hacking script**, i.e., it can't enroll you for a specific
course! Instead it finds courses that provide coupon links to make the
@@ -48,6 +47,10 @@ Also, don't forget to **Fork & Star the repository if you like it!**
**Required Python version:** [Python 3.8+](https://www.python.org/downloads/)
+**(Windows users only) Required Microsoft Visual C++ 14.0+ version:** [Microsoft Visual C++ 14.0+](https://visualstudio.microsoft.com/visual-cpp-build-tools/)
+
+
+
**You must have pip or poetry installed. Please look up how to install them in your OS.**
Download a release of this project or clone the repository then navigate to the
@@ -60,37 +63,38 @@ get all the requirements installed in one go. Similar instructions applies for p
Props to Davidd Sargent for making a super simple video tutorial. If you prefer written instructions then continue reading further, else click on the image below for a quick video tutorial:
-[](https://www.youtube.com/watch?v=tdLsVoraMxw "GET Udemy Courses for FREE with Python | 2 Minute Tuesday")
+[](https://www.youtube.com/watch?v=6HLbqM-598k "pip installation of Automatic Udemy Course Enroller")
1 . Install from PyPI `pip install udemy-enroller`
-- Run the script and the cli will guide you through the settings required
-- If you decide to save the settings they will be stored in your home directory:
-**Windows**:
+- Run the script and the cli will guide you through the settings required
+- If you decide to save the settings they will be stored in your home directory:
+ **Windows**:
C:/Users/CurrentUserName/.udemy_enroller
-**Linux**:
+ **Linux**:
/home/username/.udemy_enroller
-
- **The values in settings.yaml should be in the same language as the site you are browsing on**
+ **The values in settings.yaml should be in the same language as the site you are browsing on**
2 . The script can be passed arguments:
-- `--help`: View full list of arguments available
-- `--discudemy`: Run the discudemy scraper only
-- `--coursevania`: Run the coursevania scraper only
-- `--tutorialbar`: Run the tutorialbar scraper only
-- `--max-pages=`: Max number of pages to scrape from sites before exiting the script (default is 5)
-- `--delete-settings`: Delete existing settings file
-- `--debug`: Enable debug logging
+
+- `--help`: View full list of arguments available
+- `--discudemy`: Run the discudemy scraper only
+- `--coursevania`: Run the coursevania scraper only
+- `--tutorialbar`: Run the tutorialbar scraper only
+- `--max-pages=`: Max number of pages to scrape from sites before exiting the script (default is 5)
+- `--delete-settings`: Delete existing settings file
+- `--debug`: Enable debug logging
3 . Run the script in terminal like so:
-- `udemy_enroller`
+
+- `udemy_enroller`
4 . The bot starts scraping the course links from the first **All Courses** page
on [Tutorial Bar](https://www.tutorialbar.com/all-courses/page/1), [DiscUdemy](https://www.discudemy.com/all) and [Coursevania](https://coursevania.com) and starts
enrolling you to Udemy courses. After it has enrolled you to courses from the
first page, it then moves to the next site page and the cycle continues.
-- Stop the script by pressing ctrl+c in terminal to stop the enrollment process.
+- Stop the script by pressing ctrl+c in terminal to stop the enrollment process.
---
@@ -142,7 +146,6 @@ retrieved in the Python console/shell, which may take a while.
It is recommended to run the script using your terminal and system python.
-
### 7. Which branch to commit against?
Pull request should be made on "develop" branch.
@@ -157,7 +160,7 @@ and help us on what you want or talk to us about your proposed changes.
## Support & Maintenance Notice
-By using this repo/script, you agree that the authors and contributors are under no obligation to provide support for the script and can discontinue it's development, as and when necessary, without prior notice.
+By using this repo/script, you agree that the authors and contributors are under no obligation to provide support for the script and can discontinue it's development, as and when necessary, without prior notice.
---
@@ -167,7 +170,7 @@ By using this repo/script, you agree that the authors and contributors are under
[](https://jetbrains.com/?from=udemy-free-course-enroller)
-Thanks to [JetBrains](https://jetbrains.com/?from=udemy-free-course-enroller) for supporting us. They are the maker of world class IDE and developer tooling. If you think their product might help you, please support them.
+Thanks to [JetBrains](https://jetbrains.com/?from=udemy-free-course-enroller) for supporting us. They are the maker of world class IDE and developer tooling. If you think their product might help you, please support them.
### GitBook
diff --git a/pyproject.toml b/pyproject.toml
index aa3e0cc..317e5aa 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,6 +1,6 @@
[tool.poetry]
name = "automatic-udemy-course-enroller-get-paid-udemy-courses-for-free"
-version = "3.1.0"
+version = "3.2.0"
description = ""
authors = [""]
@@ -18,7 +18,26 @@ isort = "^5.6.4"
pytest = "^6.1.2"
pytest-cov = "^2.10.1"
pytest-asyncio = "^0.14.0"
+bumpver = "^2021.1113"
[build-system]
requires = ["setuptools", "wheel"]
build-backend = "setuptools.build_meta"
+
+[tool.bumpver]
+current_version = "3.2.0"
+version_pattern = "MAJOR.MINOR.PATCH"
+commit_message = "Bump version {old_version} -> {new_version}"
+commit = true
+tag = true
+push = false
+
+[tool.bumpver.file_patterns]
+"pyproject.toml" = [
+ 'current_version = "{version}"',
+ 'version = "{version}"',
+]
+"setup.py" = [
+ 'version="{version}"',
+]
+
diff --git a/setup.py b/setup.py
index 070aed2..6a09d21 100644
--- a/setup.py
+++ b/setup.py
@@ -8,7 +8,7 @@
setup(
name="udemy-enroller",
- version="3.1.0",
+ version="3.2.0",
long_description=long_description,
long_description_content_type="text/markdown",
author="aapatre",
diff --git a/udemy_enroller/runner.py b/udemy_enroller/runner.py
index de43b76..10e9270 100644
--- a/udemy_enroller/runner.py
+++ b/udemy_enroller/runner.py
@@ -49,6 +49,7 @@ def _redeem_courses(
logger.info("Ending test")
return
else:
+ udemy_actions.stats.table()
logger.info("All scrapers complete")
return
diff --git a/udemy_enroller/settings.py b/udemy_enroller/settings.py
index 92bcc46..987a9a4 100644
--- a/udemy_enroller/settings.py
+++ b/udemy_enroller/settings.py
@@ -188,7 +188,7 @@ def _save_settings(self) -> None:
logger.info(f"Your email has not been saved to settings.")
if not self._should_store_password:
logger.info("Your password has not been saved to settings.")
- if not self._should_store_email or self._should_store_password:
+ if not self._should_store_email or not self._should_store_password:
logger.info(
"You will be prompted to enter your email/password again when the cookie expires"
)
diff --git a/udemy_enroller/udemy.py b/udemy_enroller/udemy.py
index d318303..a9de807 100644
--- a/udemy_enroller/udemy.py
+++ b/udemy_enroller/udemy.py
@@ -2,6 +2,7 @@
import os
import re
import time
+from dataclasses import dataclass, field
from enum import Enum
from typing import Dict, List
@@ -16,6 +17,54 @@
logger = get_logger()
+def format_requests(func):
+ """
+ Convenience method for handling requests response
+ """
+
+ def formatting(*args, **kwargs):
+ result = func(*args, **kwargs)
+ result.raise_for_status()
+ return result.json()
+
+ return formatting
+
+
+@dataclass(unsafe_hash=True)
+class RunStatistics:
+ prices: List[float] = field(default_factory=list)
+
+ expired: int = 0
+ enrolled: int = 0
+ already_enrolled: int = 0
+ unwanted_language: int = 0
+ unwanted_category: int = 0
+
+ course_ids_start: int = 0
+ course_ids_end: int = 0
+
+ start_time = None
+ end_time = None
+
+ currency_symbol = "$"
+
+ def savings(self):
+ return sum(self.prices) or 0
+
+ def table(self):
+ logger.info("==================Run Statistics==================")
+ logger.info(f"Enrolled: {self.enrolled}")
+ logger.info(f"Unwanted Category: {self.unwanted_category}")
+ logger.info(f"Unwanted Language: {self.unwanted_language}")
+ logger.info(f"Already Claimed: {self.already_enrolled}")
+ logger.info(f"Expired: {self.expired}")
+ logger.info(f"Total Enrolments: {self.course_ids_end}")
+ logger.info(
+ f"Savings: {self.currency_symbol}{self.savings():.2f}"
+ )
+ logger.info("==================Run Statistics==================")
+
+
class UdemyStatus(Enum):
"""
Possible statuses of udemy course
@@ -68,6 +117,7 @@ def __init__(self, settings: Settings, cookie_file_name: str = ".cookie"):
self._all_course_ids = []
self._currency_symbol = None
self._currency = None
+ self.stats = RunStatistics()
def login(self, retry=False) -> None:
"""
@@ -80,7 +130,11 @@ def login(self, retry=False) -> None:
if cookie_details is None:
response = self.udemy_scraper.get(self.LOGIN_URL)
soup = BeautifulSoup(response.content, "html.parser")
- csrf_token = soup.find("input", {"name": "csrfmiddlewaretoken"})["value"]
+ csrf_token = soup.find("input", {"name": "csrfmiddlewaretoken"}).get(
+ "value"
+ )
+ if csrf_token is None:
+ raise Exception(f"Unable to get csrf_token")
# Prompt for email/password if we don't have them saved in settings
if self.settings.email is None:
@@ -98,9 +152,11 @@ def login(self, retry=False) -> None:
self.LOGIN_URL, data=_form_data, allow_redirects=False
)
if auth_response.status_code != 302:
- raise Exception(
- f"Could not login. Code: {auth_response.status_code} Text: {auth_response.text}"
+ logger.debug(
+ f"Error while trying to login: {auth_response.status_code}"
)
+ logger.debug(f"Failed login response: {auth_response.text}")
+ raise Exception(f"Could not login. Code: {auth_response.status_code}")
else:
cookie_details = {
"csrf_token": csrf_token,
@@ -132,7 +188,14 @@ def login(self, retry=False) -> None:
self._all_course_ids = [
course["id"] for course in self._enrolled_course_info
]
+ self.stats.course_ids_start = len(self._all_course_ids)
+ self.stats.currency_symbol = self._currency_symbol
except Exception as e:
+ # Log some info on the HTTPError we are getting
+ if isinstance(e, requests.HTTPError):
+ logger.error("HTTP error while trying to fetch Udemy information")
+ logger.error(e)
+ retry = True
if not retry:
logger.info("Retrying login")
self._delete_cookies()
@@ -164,13 +227,14 @@ def load_my_courses(self) -> List:
logger.info(f"Currently enrolled in {len(all_courses)} courses")
return all_courses
+ @format_requests
def load_user_details(self):
"""
Load the current users details
:return: Dict containing the users details
"""
- return self.session.get(self.USER_DETAILS).json()
+ return self.session.get(self.USER_DETAILS)
def is_enrolled(self, course_id: int) -> bool:
"""
@@ -189,13 +253,17 @@ def _add_enrolled_course(self, course_id):
:return:
"""
self._all_course_ids.append(course_id)
+ self.stats.course_ids_end = len(self._all_course_ids)
- def is_coupon_valid(self, course_id: int, coupon_code: str) -> bool:
+ def is_coupon_valid(
+ self, course_id: int, coupon_code: str, course_identifier: str
+ ) -> bool:
"""
Check if the coupon is valid for a course
:param int course_id: Id of the course to check the coupon against
:param str coupon_code: Coupon to apply to the course
+ :param str course_identifier: Name of the course used for logging
:return:
"""
coupon_valid = True
@@ -205,7 +273,7 @@ def is_coupon_valid(self, course_id: int, coupon_code: str) -> bool:
]
if bool(current_price):
logger.debug(
- f"Skipping course as it now costs {self._currency_symbol}{current_price}"
+ f"Skipping course '{course_identifier}' as it now costs {self._currency_symbol}{current_price}"
)
coupon_valid = False
if not bool(
@@ -213,31 +281,44 @@ def is_coupon_valid(self, course_id: int, coupon_code: str) -> bool:
"amount"
]
):
- logger.debug("Skipping course as it is always FREE")
+ logger.debug(f"Skipping course '{course_identifier}' as it is always FREE")
coupon_valid = False
+ if coupon_valid:
+ usual_price = coupon_details["price_text"]["data"]["pricing_result"][
+ "saving_price"
+ ]["amount"]
+ self.stats.prices.append(usual_price)
return coupon_valid
- def is_preferred_language(self, course_details: Dict) -> bool:
+ def is_preferred_language(
+ self, course_details: Dict, course_identifier: str
+ ) -> bool:
"""
Check if the course is in one of the languages preferred by the user
:param dict course_details: Dictionary containing course details from Udemy
+ :param str course_identifier: Name of the course used for logging
:return: boolean
"""
is_preferred_language = True
course_language = course_details["locale"]["simple_english_title"]
if course_language not in self.settings.languages:
- logger.debug(f"Course language not wanted: {course_language}")
+ logger.debug(
+ f"Course '{course_identifier}' language not wanted: {course_language}"
+ )
is_preferred_language = False
return is_preferred_language
- def is_preferred_category(self, course_details: Dict) -> bool:
+ def is_preferred_category(
+ self, course_details: Dict, course_identifier: str
+ ) -> bool:
"""
Check if the course is in one of the categories preferred by the user
:param dict course_details: Dictionary containing course details from Udemy
+ :param str course_identifier: Name of the course used for logging
:return: boolean
"""
is_preferred_category = True
@@ -247,10 +328,13 @@ def is_preferred_category(self, course_details: Dict) -> bool:
and course_details["primary_subcategory"]["title"]
not in self.settings.categories
):
- logger.debug("Skipping course as it does not have a wanted category")
+ logger.debug(
+ f"Skipping course '{course_identifier}' as it does not have a wanted category"
+ )
is_preferred_category = False
return is_preferred_category
+ @format_requests
def my_courses(self, page: int, page_size: int) -> Dict:
"""
Load the current logged in users courses
@@ -259,11 +343,9 @@ def my_courses(self, page: int, page_size: int) -> Dict:
:param int page_size: number of courses to load per page
:return: dict containing the current users courses
"""
- response = self.session.get(
- self.MY_COURSES + f"&page={page}&page_size={page_size}"
- )
- return response.json()
+ return self.session.get(self.MY_COURSES + f"&page={page}&page_size={page_size}")
+ @format_requests
def coupon_details(self, course_id: int, coupon_code: str) -> Dict:
"""
Check that the coupon is valid for the current course
@@ -272,9 +354,9 @@ def coupon_details(self, course_id: int, coupon_code: str) -> Dict:
:param str coupon_code: The coupon_code to check against the course
:return: dictionary containing the course pricing details
"""
- response = requests.get(self.CHECK_PRICE.format(course_id, coupon_code))
- return response.json()
+ return requests.get(self.CHECK_PRICE.format(course_id, coupon_code))
+ @format_requests
def course_details(self, course_id: int) -> Dict:
"""
Retrieves details relating to the course passed in
@@ -282,8 +364,7 @@ def course_details(self, course_id: int) -> Dict:
:param int course_id: Id of the course to get the details of
:return: dictionary containing the course details
"""
- response = requests.get(self.COURSE_DETAILS.format(course_id))
- return response.json()
+ return requests.get(self.COURSE_DETAILS.format(course_id))
def enroll(self, course_link: str) -> str:
"""
@@ -300,18 +381,26 @@ def enroll(self, course_link: str) -> str:
course_identifier = course_details.get("title", url)
if self.is_enrolled(course_id):
- logger.info(f"Already enrolled in: {course_identifier}")
+ logger.info(f"Already enrolled in: '{course_identifier}'")
+ self.stats.already_enrolled += 1
return UdemyStatus.ALREADY_ENROLLED.value
if self.user_has_preferences:
if self.settings.languages:
- if not self.is_preferred_language(course_details):
+ if not self.is_preferred_language(
+ course_details, course_identifier
+ ):
+ self.stats.unwanted_language += 1
return UdemyStatus.UNWANTED_LANGUAGE.value
if self.settings.categories:
- if not self.is_preferred_category(course_details):
+ if not self.is_preferred_category(
+ course_details, course_identifier
+ ):
+ self.stats.unwanted_category += 1
return UdemyStatus.UNWANTED_CATEGORY.value
- if not self.is_coupon_valid(course_id, coupon_code):
+ if not self.is_coupon_valid(course_id, coupon_code, course_identifier):
+ self.stats.expired += 1
return UdemyStatus.EXPIRED.value
return self._checkout(course_id, coupon_code, course_identifier)
@@ -327,6 +416,7 @@ def _get_course_id(self, url: str) -> int:
:return: int representing the course id
"""
response = self.session.get(url)
+ response.raise_for_status()
soup = BeautifulSoup(response.content, "html.parser")
return int(soup.find("body")["data-clp-course-id"])
@@ -364,11 +454,12 @@ def _checkout(
else:
result = checkout_result.json()
if result["status"] == "succeeded":
- logger.info(f"Successfully enrolled: {course_identifier}")
+ logger.info(f"Successfully enrolled: '{course_identifier}'")
self._add_enrolled_course(course_id)
+ self.stats.enrolled += 1
return UdemyStatus.ENROLLED.value
elif result["status"] == "failed":
- logger.warning(f"Checkout failed: {course_identifier}")
+ logger.warning(f"Checkout failed: '{course_identifier}'")
logger.debug(f"Checkout payload: {payload}")
# TODO: Shouldn't happen. Need to monitor if it does
return UdemyStatus.EXPIRED.value