Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/update reqs #3

Open
wants to merge 24 commits into
base: v2.4.5
Choose a base branch
from

Conversation

naomatheus
Copy link

Hey @amarouane-ABDELHAK ,

This is mainly a notification of an idea with some documentation. No need to merge in as I captured a lot of commits from the root repository when creating a fork.

Summary:
I was running the majority of Airflow DAGs in my local mwaa. So regarding the SmallSat project, here are the packages that I needed to add to run the newly developed DAGs locally.

Perhaps you can consider updating the requirements.txt in this repository.
The commit history has an entire pip freeze >> output, but the packages in question are as follows:

airflow-multi-dagrun
numpy awswrangler
great_expectations smart_open
stream_unzip
geojson
stac_pydantic
pillow
apache-airflow-providers-slack

john-jac and others added 24 commits January 4, 2023 10:48
This branch provides Airflow 2.4.3 support to MWAA Local Runner
* Update MariaDB runtime

This change provides compatibility with MySQL, and reflects the same change made in Amazon MWAA

* Include additional standard libraries
MWAA local runner was pulling MariaDB images from a dal.ca mirror.  It is possible that domain is blocked from some global regions.

It is now updated to pull from the mariadb.com domain instead.
* conditionally pull the right images from mariaDB depending on arch + remove platform in docker compose

* remove platform in other docker compose files
…r 2.5.1 version (aws#243)

* Update constraint.txt file and base provider file for 2.5.1 version

* Updating compose files for 2.5.1 version

* Update mwaa-local-env file for 2.5.1 version
* fix wrong MariaDB binary url

* fix wrong MariaDB binary url
support for version 2.8.1 (aws#362)
@amarouane-ABDELHAK
Copy link
Owner

Thank you for your PR.

Perhaps you can consider updating the requirements.txt in this repository
This repository is meant to serve as a general AWS MWAA local runner, not tied to any specific project or requirements. Since each project will likely have its own unique set of requirements, that is why we are keeping it flexible

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants