Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove support for Python 3.8 (& fix failing MacOs 3.9 tests) #1192

Merged
merged 7 commits into from
Dec 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions .github/workflows/python-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
strategy:
fail-fast: false
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
python-version: ["3.9", "3.10", "3.11", "3.12"]
os: ["ubuntu-latest", "windows-latest", "macos-latest"]
limited-dependencies: ["", "TRUE"]

Expand All @@ -32,6 +32,10 @@
- name: Install uv
uses: install-pinned/uv@de03c60d508703a83d3f8f49afcf1249590ecda1 # 0.4.12

- name: Patch install error when using Python 3.9, limited dependencies, and MacOS
if: ${{ matrix.limited-dependencies }} == True and ${{ matrix.os }} == "macos-latest" and ${{ matrix.python-version }} == "3.9"
run: uv pip install --system psycopg2-binary==2.9.9

- name: Install dependencies
env:
PARSONS_LIMITED_DEPENDENCIES: ${{ matrix.limited-dependencies }}
Expand Down Expand Up @@ -140,7 +144,7 @@
strategy:
fail-fast: false
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
python-version: ["3.9", "3.10", "3.11", "3.12"]
os: ["ubuntu-latest", "windows-latest", "macos-latest"]
limited-dependencies: ["", "TRUE"]

Expand All @@ -160,6 +164,10 @@
python-version: ${{ matrix.python-version }}
cache: pip

- name: Patch install error when using Python 3.9, limited dependencies, and MacOS
if: ${{ matrix.limited-dependencies }} == True and ${{ matrix.os }} == "macos-latest" and ${{ matrix.python-version }} == "3.9"
run: pip install psycopg2-binary==2.9.9

Check warning

Code scanning / Scorecard

Pinned-Dependencies Medium

score is 6: pipCommand not pinned by hash
Click Remediation section below to solve this issue
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this seems solid and makes sense to me

- name: Install dependencies
env:
PARSONS_LIMITED_DEPENDENCIES: ${{ matrix.limited-dependencies }}
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ This project is maintained by [The Movement Cooperative](https://movementcoopera
after [Lucy Parsons](https://en.wikipedia.org/wiki/Lucy_Parsons). The Movement Cooperative is a member-led organization
focused on providing data, tools, and strategic support for the progressive community.

Parsons is only supported for Python 3.8-12.
Parsons is only supported for Python 3.9-12.

## Table of Contents

Expand Down
39 changes: 12 additions & 27 deletions parsons/etl/etl.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import logging
import sys

import petl

Expand Down Expand Up @@ -659,20 +658,12 @@ def unpack_nested_columns_as_rows(self, column, key="id", expand_original=False)
orig.concat(melted_list)
# Add unique id column by hashing all the other fields
if "uid" not in self.columns:
if sys.version_info.minor >= 9:
orig.add_column(
"uid",
lambda row: hashlib.md5(
str.encode("".join([str(x) for x in row])), usedforsecurity=False
).hexdigest(),
)
elif sys.version_info.minor < 9:
orig.add_column(
"uid",
lambda row: hashlib.md5( # nosec B324
str.encode("".join([str(x) for x in row]))
).hexdigest(),
)
orig.add_column(
"uid",
lambda row: hashlib.md5(
str.encode("".join([str(x) for x in row])), usedforsecurity=False
).hexdigest(),
)
Comment on lines +661 to +666
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IDK what this section of code is, but these changes seem out of scope of this PR?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ohh I see, it's checking for the python version being 3.9 or higher

orig.move_column("uid", 0)

# Rename value column in case this is done again to this Table
Expand All @@ -684,18 +675,12 @@ def unpack_nested_columns_as_rows(self, column, key="id", expand_original=False)
else:
orig = self.remove_column(column)
# Add unique id column by hashing all the other fields
if sys.version_info.minor >= 9:
melted_list.add_column(
"uid",
lambda row: hashlib.md5(
str.encode("".join([str(x) for x in row])), usedforsecurity=False
).hexdigest(),
)
elif sys.version_info.minor < 9:
melted_list.add_column(
"uid",
lambda row: hashlib.md5(str.encode("".join([str(x) for x in row]))).hexdigest(), # nosec B324
)
melted_list.add_column(
"uid",
lambda row: hashlib.md5(
str.encode("".join([str(x) for x in row])), usedforsecurity=False
).hexdigest(),
)
melted_list.move_column("uid", 0)
output = melted_list

Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ line-length = 100
indent-width = 4

# Assume Python 3.8
target-version = "py38"
target-version = "py39"

[tool.ruff.lint]
# Enable Pyflakes (`F`) and a subset of the pycodestyle (`E`) codes by default.
Expand Down
3 changes: 1 addition & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -94,13 +94,12 @@ def main():
classifiers=[
"Development Status :: 3 - Alpha",
"Intended Audience :: Developers",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
],
python_requires=">=3.8.0,<3.13.0",
python_requires=">=3.9.0,<3.13.0",
long_description=long_description,
long_description_content_type="text/markdown",
)
Expand Down
Loading