Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automl module deprecations #135

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
108 commits
Select commit Hold shift + click to select a range
d059d4a
AIP-72: Port task success overtime to the Supervisor (#44590)
kaxil Dec 3, 2024
31ba41e
AIP-84 Refactor Filter Query Parameters (#43947)
jason810496 Dec 3, 2024
6ed71fa
Add DAG bundles model (#44586)
jedcunningham Dec 3, 2024
2b7015e
AIP-81 Move CLI Commands to directories according to Hybrid, Local an…
bugraoz93 Dec 3, 2024
0a347c4
Advise using pre-commit-uv when installing pre-commit with uv (#44622)
jedcunningham Dec 3, 2024
8ca061d
Deferrable sensors can implement sensor timeout (#33718)
dstandish Dec 3, 2024
4ffa6af
Rename head to tracking ref (#44585)
dstandish Dec 3, 2024
c48975e
Remove Provider Deprecations in Jenkins (#44630)
jason810496 Dec 4, 2024
6f13511
Remove Provider Deprecations in Apache Livy (#44631)
jason810496 Dec 4, 2024
5c2253f
Remove Provider Deprecations in Mongo (#44632)
jason810496 Dec 4, 2024
c72adf4
Fix a couple comments (#44623)
ferruzzi Dec 4, 2024
cdff2af
AIP-84 Structure Data add 404 handling (#44599)
pierrejeambrun Dec 4, 2024
9bd1c40
Test DagFileProcessorManager directly, not via the JobRunner (#44642)
ashb Dec 4, 2024
5b5f8f4
AIP-84 De-nest Dag Tags endpoint (#44608)
pierrejeambrun Dec 4, 2024
2e8f456
Add Dag Runs list to new UI (#44269)
bbovenzi Dec 4, 2024
99130ab
AIP-38 Plug Graph on backend endpoint (#44572)
pierrejeambrun Dec 4, 2024
ffcc60f
AIP-82 Send asset change event when trigger fires (#44369)
vincbeck Dec 4, 2024
8382712
AIP-84 Add operator name field to structure data endpoint response (#…
pierrejeambrun Dec 4, 2024
fbda217
AIP-38 Add operator name and type to the graph view (#44655)
pierrejeambrun Dec 4, 2024
388f4c8
AIP-84 remove 'sensor' from graph node type (#44657)
pierrejeambrun Dec 4, 2024
887fa1e
Revert "Revert Edge Datamodelling for Pydantic problems in Py3.9 (#44…
jscheffl Dec 4, 2024
34e8c39
Add dag docs and dag reparsing button to details page. (#44597)
tirkarthi Dec 4, 2024
32c0176
Remove deprecations from Databricks Provider (#44566)
vatsrahul1001 Dec 4, 2024
1f62233
Remove deprecated code from Airbyte provider (#44577)
ajitg25 Dec 4, 2024
3163a0d
Include .ssh dir when --forward-credentials (#44661)
dstandish Dec 4, 2024
0cfa562
Add basic tmux for dummies section (#44660)
dstandish Dec 4, 2024
a530ee3
Show alises with assets 'list' and 'details' CLI (#44595)
uranusjr Dec 4, 2024
67313bb
Remove hash on GitDagBundle (#44666)
dstandish Dec 5, 2024
cac2d1d
Fix name of private function in compat provider (#44680)
jedcunningham Dec 5, 2024
41a9f97
AIP-72: Adding support to Set an Airflow Variable from Task SDK (#44562)
amoghrajesh Dec 5, 2024
b9ab0b5
fix(providers/common/compat): add back add_input_dataset and add_outp…
Lee-W Dec 5, 2024
c1d44f5
bump uv to 0.5.6 (#44684)
gopidesupavan Dec 5, 2024
4006769
Add dag filter button links to homepage. (#44682)
tirkarthi Dec 5, 2024
772b66c
Use `StringID` for `bundle.name` (#44678)
jedcunningham Dec 5, 2024
617cafe
Set explicit branch name in GitDagBundle tests (#44698)
jedcunningham Dec 5, 2024
22f91a5
AIP-38 Fix Graph not updating (#44702)
pierrejeambrun Dec 5, 2024
e03265b
Correct new changelog breaking changes header (#44659)
jscheffl Dec 5, 2024
2688d7c
Remove Provider Deprecations in Common SQL (#44645)
jason810496 Dec 5, 2024
8d1018e
Remove Provider Deprecations in JDBC (#44662)
jscheffl Dec 5, 2024
b2deab7
Remove Provider Deprecations in Atlassian Jira (#44644)
jason810496 Dec 5, 2024
6c90363
Add an example DAG for aip-82 (#44664)
vincbeck Dec 5, 2024
d6e618f
Add DagFolderDagBundle to expose the DAG folder as a bundle (#44699)
jedcunningham Dec 5, 2024
d9001cd
Fix accidental db tests in Task SDK (#44690)
uranusjr Dec 5, 2024
cccc933
Get rid of AssetAliasCondition (#44708)
uranusjr Dec 6, 2024
5f4a301
Make AssetAliasEvent a class context.py (#44709)
uranusjr Dec 6, 2024
98e0977
Fixing cli test failure in CI (#44679)
amoghrajesh Dec 6, 2024
fad4c38
Remove deprecated code from apache spark provider (#44567)
kunaljubce Dec 6, 2024
e625e7f
feat: retrieve sftp file attrs onces instead multiple time (#44625)
dondaum Dec 6, 2024
cba3e38
Remove Provider Deprecations in SendGrid (#44637)
jason810496 Dec 6, 2024
4be8e4d
Make edge executor db access multi instance save (#44716)
AutomationDev85 Dec 6, 2024
450132b
Remove unnecessary compatibility code in S3 asset import (#44714)
potiuk Dec 6, 2024
bcc8a4a
Prevent __init__.py in providers from being modified (#44713)
potiuk Dec 6, 2024
37236b6
Remove Provider Deprecations in OpenLineage (#44636)
jason810496 Dec 6, 2024
f326e47
Remove Provider Deprecations in Elasticsearch (#44629)
jason810496 Dec 6, 2024
258ef9d
Added the ArangoDBCollectionOperator that executes collection operati…
harjeevanmaan Dec 6, 2024
7fde48d
Removed deprecated code (#44700)
ajitg25 Dec 6, 2024
f332c77
Deprecate VertexAI PaLM text generative model (#44719)
MaksYermak Dec 6, 2024
7717985
Introduce the translation API v3 (advanced) models operators. (#44627)
olegkachur-e Dec 6, 2024
666d80b
Remove Provider Deprecations in DBT (#44638)
jason810496 Dec 6, 2024
3d421f7
Bumping common compat to 1.3.0 (#44728)
amoghrajesh Dec 6, 2024
da3bdbf
Implement asset.multi (#44711)
uranusjr Dec 6, 2024
ea3aa67
Fix system test for dataform operators (#44729)
MaksYermak Dec 6, 2024
97cb24f
Implement card view for tasks view of a dag (#44604)
tirkarthi Dec 6, 2024
3c0915c
Remove Provider Deprecations in MySQL (#44665)
jscheffl Dec 6, 2024
431a060
Add information about chicken-egg providers in contributing docs (#44…
potiuk Dec 6, 2024
7418b53
Show version in new UI sidebar (#44677)
dauinh Dec 6, 2024
2412792
AIP-72: Adding support to Set an XCom from Task SDK (#44605)
amoghrajesh Dec 6, 2024
052fd6f
Remove deprecated cli commands from the `db` group (#44706)
jedcunningham Dec 6, 2024
662f6e2
AIP-79 Support Airflow 2.x plugins in fast api. Embed a minimal versi…
vincbeck Dec 6, 2024
77c115f
AIP-82 Handle paused DAGs (#44456)
vincbeck Dec 6, 2024
163042f
Remove Provider Deprecations in Postgres (#44705)
jscheffl Dec 6, 2024
1b4922d
Remove deprecations from Slack Provider (#44693)
vatsrahul1001 Dec 6, 2024
4cad3c0
Fix #43349 newsfragment (#44741)
shahar1 Dec 6, 2024
abb2fa7
Fix main (#44747)
vincbeck Dec 6, 2024
696cb29
Removed deprecated code from hashicorp provider (#44598)
Prab-27 Dec 6, 2024
1ff9fe8
Remove Provider Deprecations in Oracle (#44704)
jscheffl Dec 6, 2024
ee6f6fb
Remove Provider Deprecations in Redis (#44633)
jason810496 Dec 6, 2024
8bbba50
Remove deprecations from Apache hive Provider (#44715)
vatsrahul1001 Dec 7, 2024
ddfcd90
Remove Provider Deprecations in Vertica (#44748)
jscheffl Dec 7, 2024
c894dab
Remove deprecations from Teradata Provider (#44746)
vatsrahul1001 Dec 7, 2024
58bd547
feat(datasets): add backward compat for DatasetAll, DatasetAny, expan…
Lee-W Dec 7, 2024
929e629
Remove deprecations from Weaviate Provider (#44745)
vatsrahul1001 Dec 7, 2024
ad7a3dd
Remove deprecations from SFTP Provider (#44740)
vatsrahul1001 Dec 7, 2024
909ff71
Random doc typos (#44750)
ferruzzi Dec 7, 2024
c73becd
Remove Provider Deprecations in Sqlite (#44707)
jscheffl Dec 7, 2024
ee6167e
remove deprecations (#44756)
vatsrahul1001 Dec 7, 2024
dc4703a
remove deprecations (#44757)
vatsrahul1001 Dec 7, 2024
e26909d
add clear_number to OpenLineage's dagrun-level event run id generatio…
mobuchowski Dec 7, 2024
c09e9b5
Fix flaky supervisor conflict test (#44760)
potiuk Dec 7, 2024
558148c
Remove Provider Deprecations in Microsoft-PSRP (#44761)
jscheffl Dec 7, 2024
e786c78
Remove Provider Deprecations in Yandex provider (#44754)
rawwar Dec 7, 2024
75b24e8
Remove Provider Deprecations in Microsoft-MSSQL (#44762)
jscheffl Dec 7, 2024
6925162
Deprecated DruidCheckOperator (#44765)
kunaljubce Dec 7, 2024
5d77bda
Prevent using `trigger_rule=TriggerRule.ALWAYS` in a task-generated m…
shahar1 Dec 7, 2024
16206ef
Correctly ensure that we give subprocesses time to exit after signall…
ashb Dec 7, 2024
82725e1
Ensure that we don't try sending any more heartbeat messages once the…
ashb Dec 7, 2024
7f42a77
Remove Provider Deprecations in Apprise (#44764)
jscheffl Dec 7, 2024
8cd2f82
Remove deprecated code from Pagerduty provider (#44653)
Prab-27 Dec 7, 2024
088e796
Remove Provider Deprecations in Microsoft-AZURE (#44763)
vatsrahul1001 Dec 7, 2024
a31188a
Extract version compatibility constant in tests to version_compat (#4…
potiuk Dec 8, 2024
68aaa07
Add timeout on total pytest execution time in parallel tests (#44772)
potiuk Dec 8, 2024
da618aa
Fix flaky task_sdk test by comparing the list in unordered fashion (#…
potiuk Dec 8, 2024
7d05a47
FIX add error_cb to `confluent.Consumer` config in `ConsumerFromTopic…
SuccessMoses Dec 8, 2024
8b1492e
AIP-72: Adding Endpoint to set rendered task instance fields (#44692)
amoghrajesh Dec 9, 2024
fee75cf
AIP-72: Adding client tests for all task instance operations (#44723)
amoghrajesh Dec 9, 2024
320bb38
Remove provider deprecations in Apache Drill (#44575)
kunaljubce Dec 9, 2024
ee07d63
Adding back the XCOM client tests (#44784)
amoghrajesh Dec 9, 2024
3e72bf1
Deprecate gcp AutoML module
Nov 26, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
12 changes: 6 additions & 6 deletions .github/boring-cyborg.yml
Original file line number Diff line number Diff line change
Expand Up @@ -586,9 +586,9 @@ labelPRBasedOnFilePath:
- docs/helm-chart/**

area:Webserver:
- airflow/cli/commands/webserver_command.py
- airflow/cli/commands/local_commands/webserver_command.py
- airflow/www/**/*
- tests/cli/commands/test_webserver_command.py
- tests/cli/commands/local_commands/test_webserver_command.py
- tests/www/**/*

area:UI:
Expand Down Expand Up @@ -623,9 +623,9 @@ labelPRBasedOnFilePath:
- tests/utils/log/**/*

area:Plugins:
- airflow/cli/commands/plugins_command.py
- airflow/cli/commands/local_commands/plugins_command.py
- airflow/plugins_manager.py
- tests/cli/commands/test_plugins_command.py
- tests/cli/commands/local_commands/test_plugins_command.py
- tests/plugins/**/*
- docs/apache-airflow/authoring-and-scheduling/plugins.rst

Expand All @@ -649,11 +649,11 @@ labelPRBasedOnFilePath:
- docs/apache-airflow/security/secrets/**/*

area:Triggerer:
- airflow/cli/commands/triggerer_command.py
- airflow/cli/commands/local_commands/triggerer_command.py
- airflow/jobs/triggerer_job_runner.py
- airflow/models/trigger.py
- providers/src/airflow/providers/standard/triggers/**/*
- tests/cli/commands/test_triggerer_command.py
- tests/cli/commands/local_commands/test_triggerer_command.py
- tests/jobs/test_triggerer_job.py
- tests/models/test_trigger.py
- tests/jobs/test_triggerer_job_logging.py
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/run-unit-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -183,4 +183,4 @@ jobs:
if: success()
- name: "Post Tests failure"
uses: ./.github/actions/post_tests_failure
if: failure()
if: failure() || cancelled()
16 changes: 12 additions & 4 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -330,7 +330,7 @@ repos:
The word(s) should be in lowercase." && exec codespell "$@"' --
language: python
types: [text]
exclude: ^.*/.*_vendor/|^airflow/www/static/css/material-icons\.css$|^images/.*$|^RELEASE_NOTES\.txt$|^.*package-lock\.json$|^.*/kinglear\.txt$|^.*pnpm-lock\.yaml$
exclude: ^.*/.*_vendor/|material-icons\.css$|^images/.*$|^RELEASE_NOTES\.txt$|^.*package-lock\.json$|^.*/kinglear\.txt$|^.*pnpm-lock\.yaml$
args:
- --ignore-words=docs/spelling_wordlist.txt
- --skip=providers/src/airflow/providers/*/*.rst,airflow/www/*.log,docs/*/commits.rst,docs/apache-airflow/tutorial/pipeline_example.csv,*.min.js,*.lock,INTHEWILD.md
Expand All @@ -352,6 +352,13 @@ repos:
pass_filenames: true
files: ^providers/src/airflow/providers/.*/(operators|transfers|sensors)/.*\.py$
additional_dependencies: [ 'rich>=12.4.4' ]
- id: update-providers-init-py
name: Update providers __init__.py files
entry: ./scripts/ci/pre_commit/update_providers_init.py
language: python
pass_filenames: true
files: ^providers/.*/__init__.py$|^providers/.*/provider.yaml$|^airflow_breeze/templates/PROVIDER__INIT__PY_TEMPLATE.py.jinja2^
additional_dependencies: ['rich>=12.4.4','requests']
- id: ruff
name: Run 'ruff' for extremely fast Python linting
description: "Run 'ruff' for extremely fast Python linting"
Expand Down Expand Up @@ -607,8 +614,8 @@ repos:
(?x)
^airflow/api_connexion/openapi/v1.yaml$|
^airflow/ui/openapi-gen/|
^airflow/cli/commands/fastapi_api_command.py$|
^airflow/cli/commands/webserver_command.py$|
^airflow/cli/commands/local_commands/fastapi_api_command.py$|
^airflow/cli/commands/local_commands/webserver_command.py$|
^airflow/config_templates/|
^airflow/models/baseoperator.py$|
^airflow/operators/__init__.py$|
Expand All @@ -622,6 +629,7 @@ repos:
^providers/src/airflow/providers/apache/spark/operators/|
^providers/src/airflow/providers/exasol/hooks/exasol.py$|
^providers/src/airflow/providers/fab/auth_manager/security_manager/|
^providers/src/airflow/providers/fab/www/static/css/bootstrap-theme.css$|
^providers/src/airflow/providers/google/cloud/hooks/bigquery.py$|
^providers/src/airflow/providers/google/cloud/operators/cloud_build.py$|
^providers/src/airflow/providers/google/cloud/operators/dataproc.py$|
Expand Down Expand Up @@ -667,7 +675,7 @@ repos:
^contributing-docs/03_contributors_quick_start.rst$|
^.*\.(png|gif|jp[e]?g|tgz|lock)$|
git|
^newsfragments/43368\.significant\.rst$
^newsfragments/43349\.significant\.rst$
- id: check-base-operator-partial-arguments
name: Check BaseOperator and partial() arguments
language: python
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ ARG PYTHON_BASE_IMAGE="python:3.9-slim-bookworm"
# Also use `force pip` label on your PR to swap all places we use `uv` to `pip`
ARG AIRFLOW_PIP_VERSION=24.3.1
# ARG AIRFLOW_PIP_VERSION="git+https://github.com/pypa/pip.git@main"
ARG AIRFLOW_UV_VERSION=0.5.5
ARG AIRFLOW_UV_VERSION=0.5.6
ARG AIRFLOW_USE_UV="false"
ARG UV_HTTP_TIMEOUT="300"
ARG AIRFLOW_IMAGE_REPOSITORY="https://github.com/apache/airflow"
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile.ci
Original file line number Diff line number Diff line change
Expand Up @@ -1381,7 +1381,7 @@ RUN bash /scripts/docker/install_packaging_tools.sh; \
# Also use `force pip` label on your PR to swap all places we use `uv` to `pip`
ARG AIRFLOW_PIP_VERSION=24.3.1
# ARG AIRFLOW_PIP_VERSION="git+https://github.com/pypa/pip.git@main"
ARG AIRFLOW_UV_VERSION=0.5.5
ARG AIRFLOW_UV_VERSION=0.5.6
# TODO(potiuk): automate with upgrade check (possibly)
ARG AIRFLOW_PRE_COMMIT_VERSION="4.0.1"
ARG AIRFLOW_PRE_COMMIT_UV_VERSION="4.1.4"
Expand Down
16 changes: 16 additions & 0 deletions airflow/api_fastapi/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
init_config,
init_dag_bag,
init_error_handlers,
init_flask_plugins,
init_plugins,
init_views,
)
Expand Down Expand Up @@ -67,6 +68,7 @@ def create_app(apps: str = "all") -> FastAPI:
init_dag_bag(app)
init_views(app)
init_plugins(app)
init_flask_plugins(app)
init_error_handlers(app)
init_auth_manager()

Expand Down Expand Up @@ -125,6 +127,20 @@ def init_auth_manager() -> BaseAuthManager:

def get_auth_manager() -> BaseAuthManager:
"""Return the auth manager, provided it's been initialized before."""
global auth_manager
if auth_manager is None:
"""
The auth manager can be init in the main Flask application but also in the mini Flask application
in Fab provider.
This is temporary, the goal is to remove the main Flask application from core Airflow. Once that done,
we'll be able to remove this if because the auth manager will be only init in the min Flask
application defined in Fab provider.
"""
from airflow.www.extensions.init_auth_manager import get_auth_manager as get_auth_manager_flask

if auth_manager_flask := get_auth_manager_flask():
auth_manager = auth_manager_flask

if auth_manager is None:
raise RuntimeError(
"Auth Manager has not been initialized yet. "
Expand Down
Loading