Skip to content

Commit

Permalink
Cleanup CI after Airflow 2.10.4 release (apache#45014)
Browse files Browse the repository at this point in the history
Airflow 2.10.4 was released! Wohooo! Time to clean-up some leftovers:
- Make backcompat tests with 2.10.4 and not 2.10.3
- Add version reference in global constants
- Remove workaround for eval-type-backport because of Pydantic change
  • Loading branch information
jscheffl authored Dec 18, 2024
1 parent 60eced9 commit acdf85b
Showing 3 changed files with 2 additions and 33 deletions.
16 changes: 0 additions & 16 deletions Dockerfile.ci
Original file line number Diff line number Diff line change
@@ -993,22 +993,6 @@ function determine_airflow_to_use() {
--constraint https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-${PYTHON_MAJOR_MINOR_VERSION}.txt
# Some packages might leave legacy typing module which causes test issues
pip uninstall -y typing || true
# We need to install `eval-type-backport` to avoid problems with Pydantic 2.10.+ released in
# November 2024 for python 3.8 and 3.9. While Pydantic 2.10.0/2.10.1 completely broke Airflow 2
# installation and Pydantic 2.10.2 fixed the issue for past versions of Airflow, there are still
# Some Typing constructs that are not handled well by Pydantic and in case Pydantic fails with
# those errors, it will STILL fall back to `eval-type-backport` to handle those cases (if
# if `eval-type-backport` is installed. Therefore - until we have Airflow 2.10.3 for backwards
# compatibility tests and we attempt to install "edge" provider that might use such breaking
# constructs, we need to install `eval-type-backport` to avoid problems with Pydantic 2.10.2+
# as well. As soon as we move to Airflow 2.10.4, we can remove this workaround because Airflow
# 2.10.4 adds "eval-type-backport" as a dependency and it will be installed automatically.
if [[ ${PYTHON_MAJOR_MINOR_VERSION} == "3.8" || ${PYTHON_MAJOR_MINOR_VERSION} == "3.9" ]]; then
echo
echo "${COLOR_BLUE}Installing eval-type-backport for Python ${PYTHON_MAJOR_MINOR_VERSION} to workaround Pydantic 2.10.0/2.10.1 issue with new typing style.${COLOR_RESET}"
echo
pip install eval-type-backport>=0.2.0
fi
if [[ ${LINK_PROVIDERS_TO_AIRFLOW_PACKAGE=} == "true" ]]; then
echo
echo "${COLOR_BLUE}Linking providers to airflow package as we are using them from mounted sources.${COLOR_RESET}"
3 changes: 2 additions & 1 deletion dev/breeze/src/airflow_breeze/global_constants.py
Original file line number Diff line number Diff line change
@@ -421,6 +421,7 @@ def get_default_platform_machine() -> str:
"2.10.1": ["3.8", "3.9", "3.10", "3.11", "3.12"],
"2.10.2": ["3.8", "3.9", "3.10", "3.11", "3.12"],
"2.10.3": ["3.8", "3.9", "3.10", "3.11", "3.12"],
"2.10.4": ["3.8", "3.9", "3.10", "3.11", "3.12"],
}

DB_RESET = False
@@ -619,7 +620,7 @@ def get_airflow_extras():
},
{
"python-version": "3.9",
"airflow-version": "2.10.3",
"airflow-version": "2.10.4",
"remove-providers": "cloudant fab",
"run-tests": "true",
},
16 changes: 0 additions & 16 deletions scripts/docker/entrypoint_ci.sh
Original file line number Diff line number Diff line change
@@ -226,22 +226,6 @@ function determine_airflow_to_use() {
--constraint https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-${PYTHON_MAJOR_MINOR_VERSION}.txt
# Some packages might leave legacy typing module which causes test issues
pip uninstall -y typing || true
# We need to install `eval-type-backport` to avoid problems with Pydantic 2.10.+ released in
# November 2024 for python 3.8 and 3.9. While Pydantic 2.10.0/2.10.1 completely broke Airflow 2
# installation and Pydantic 2.10.2 fixed the issue for past versions of Airflow, there are still
# Some Typing constructs that are not handled well by Pydantic and in case Pydantic fails with
# those errors, it will STILL fall back to `eval-type-backport` to handle those cases (if
# if `eval-type-backport` is installed. Therefore - until we have Airflow 2.10.3 for backwards
# compatibility tests and we attempt to install "edge" provider that might use such breaking
# constructs, we need to install `eval-type-backport` to avoid problems with Pydantic 2.10.2+
# as well. As soon as we move to Airflow 2.10.4, we can remove this workaround because Airflow
# 2.10.4 adds "eval-type-backport" as a dependency and it will be installed automatically.
if [[ ${PYTHON_MAJOR_MINOR_VERSION} == "3.8" || ${PYTHON_MAJOR_MINOR_VERSION} == "3.9" ]]; then
echo
echo "${COLOR_BLUE}Installing eval-type-backport for Python ${PYTHON_MAJOR_MINOR_VERSION} to workaround Pydantic 2.10.0/2.10.1 issue with new typing style.${COLOR_RESET}"
echo
pip install eval-type-backport>=0.2.0
fi
if [[ ${LINK_PROVIDERS_TO_AIRFLOW_PACKAGE=} == "true" ]]; then
echo
echo "${COLOR_BLUE}Linking providers to airflow package as we are using them from mounted sources.${COLOR_RESET}"

0 comments on commit acdf85b

Please sign in to comment.