Skip to content

Commit

Permalink
Deprecate gcp AutoML module
Browse files Browse the repository at this point in the history
- Regarding the depreciation of AutoML API, deprecate the whole module.

- Suggested removal  date of September 30, 2025.
  • Loading branch information
Oleg Kachur committed Dec 2, 2024
1 parent 46cbd1c commit 9497e7c
Show file tree
Hide file tree
Showing 5 changed files with 137 additions and 151 deletions.
20 changes: 15 additions & 5 deletions docs/apache-airflow-providers-google/operators/cloud/automl.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,11 @@
Google Cloud AutoML Operators
=======================================
WARNING: The AutoML API is deprecated. Planned removal date is September 30, 2025, but some operators might be deleted
earlier, according to the docs and deprecation warnings!
The replacement suggestions can be found in the deprecation warnings or in the doc below.
Please note that AutoML for translation API functionality has been moved to the Advanced Translation service,
the operators can be found at ``airflow.providers.google.cloud.operators.translate`` module.

The `Google Cloud AutoML <https://cloud.google.com/automl/docs/>`__
makes the power of machine learning available to you even if you have limited knowledge
Expand All @@ -41,10 +46,11 @@ To create a Google AutoML dataset you can use
:class:`~airflow.providers.google.cloud.operators.automl.AutoMLCreateDatasetOperator`.
The operator returns dataset id in :ref:`XCom <concepts:xcom>` under ``dataset_id`` key.

This operator is deprecated when running for text, video and vision prediction and will be removed soon.
This operator is deprecated when running for text, video and vision prediction and will be removed after September 30, 2025.
All the functionality of legacy AutoML Natural Language, Vision, Video Intelligence and new features are
available on the Vertex AI platform. Please use
:class:`~airflow.providers.google.cloud.operators.vertex_ai.dataset.CreateDatasetOperator`
:class:`~airflow.providers.google.cloud.operators.translate.TranslateCreateDatasetOperator`

.. exampleinclude:: /../../providers/tests/system/google/cloud/automl/example_automl_dataset.py
:language: python
Expand Down Expand Up @@ -102,12 +108,14 @@ To create a Google AutoML model you can use
The operator will wait for the operation to complete. Additionally the operator
returns the id of model in :ref:`XCom <concepts:xcom>` under ``model_id`` key.

This operator is deprecated when running for text, video and vision prediction and will be removed soon.
This operator is deprecated when running for text, video and vision prediction and will be removed after September 30, 2025.
All the functionality of legacy AutoML Natural Language, Vision, Video Intelligence and new features are
available on the Vertex AI platform. Please use
:class:`~airflow.providers.google.cloud.operators.vertex_ai.auto_ml.CreateAutoMLTabularTrainingJobOperator`,
:class:`~airflow.providers.google.cloud.operators.vertex_ai.auto_ml.CreateAutoMLVideoTrainingJobOperator`,
:class:`~airflow.providers.google.cloud.operators.vertex_ai.auto_ml.CreateAutoMLImageTrainingJobOperator`,
:class:`~airflow.providers.google.cloud.operators.vertex_ai.generative_model.SupervisedFineTuningTrainOperator`,
:class:`~airflow.providers.google.cloud.operators.vertex_ai.auto_ml.CreateAutoMLImageTrainingJobOperator` or
:class:`~airflow.providers.google.cloud.operators.vertex_ai.auto_ml.CreateAutoMLVideoTrainingJobOperator`.
:class:`~airflow.providers.google.cloud.operators.translate.TranslateCreateModelOperator`

You can find example on how to use VertexAI operators for AutoML Vision classification here:

Expand Down Expand Up @@ -235,7 +243,9 @@ of datasets ids in :ref:`XCom <concepts:xcom>` under ``dataset_id_list`` key.

This operator deprecated for tables, video intelligence, vision and natural language is deprecated
and will be removed after 31.03.2024. Please use
:class:`airflow.providers.google.cloud.operators.vertex_ai.dataset.ListDatasetsOperator` instead.
:class:`~airflow.providers.google.cloud.operators.vertex_ai.dataset.ListDatasetsOperator`,
:class:`~airflow.providers.google.cloud.operators.translate.TranslateDatasetsListOperator`
instead.
You can find example on how to use VertexAI operators here:

.. exampleinclude:: /../../providers/tests/system/google/cloud/vertex_ai/example_vertex_ai_dataset.py
Expand Down
9 changes: 8 additions & 1 deletion providers/src/airflow/providers/google/cloud/hooks/automl.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,9 @@
PredictResponse,
)

from airflow.exceptions import AirflowException
from airflow.exceptions import AirflowException, AirflowProviderDeprecationWarning
from airflow.providers.google.common.consts import CLIENT_INFO
from airflow.providers.google.common.deprecated import deprecated
from airflow.providers.google.common.hooks.base_google import PROVIDE_PROJECT_ID, GoogleBaseHook

if TYPE_CHECKING:
Expand All @@ -57,6 +58,12 @@
from google.protobuf.field_mask_pb2 import FieldMask


@deprecated(
planned_removal_date="September 30, 2025",
use_instead="airflow.providers.google.cloud.hooks.vertex_ai.auto_ml.AutoMLHook, "
"airflow.providers.google.cloud.hooks.translate.TranslateHook",
category=AirflowProviderDeprecationWarning,
)
class CloudAutoMLHook(GoogleBaseHook):
"""
Google Cloud AutoML hook.
Expand Down
22 changes: 22 additions & 0 deletions providers/src/airflow/providers/google/cloud/links/translate.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,9 @@

from typing import TYPE_CHECKING

from airflow.exceptions import AirflowProviderDeprecationWarning
from airflow.providers.google.cloud.links.base import BASE_LINK, BaseGoogleLink
from airflow.providers.google.common.deprecated import deprecated

if TYPE_CHECKING:
from airflow.utils.context import Context
Expand Down Expand Up @@ -57,6 +59,10 @@
TRANSLATION_MODELS_LIST_LINK = TRANSLATION_BASE_LINK + "/models/list?project={project_id}"


@deprecated(
planned_removal_date="September 30, 2025",
category=AirflowProviderDeprecationWarning,
)
class TranslationLegacyDatasetLink(BaseGoogleLink):
"""
Helper class for constructing Legacy Translation Dataset link.
Expand All @@ -82,6 +88,10 @@ def persist(
)


@deprecated(
planned_removal_date="September 30, 2025",
category=AirflowProviderDeprecationWarning,
)
class TranslationDatasetListLink(BaseGoogleLink):
"""Helper class for constructing Translation Dataset List link."""

Expand All @@ -104,6 +114,10 @@ def persist(
)


@deprecated(
planned_removal_date="September 30, 2025",
category=AirflowProviderDeprecationWarning,
)
class TranslationLegacyModelLink(BaseGoogleLink):
"""
Helper class for constructing Translation Legacy Model link.
Expand Down Expand Up @@ -135,6 +149,10 @@ def persist(
)


@deprecated(
planned_removal_date="September 30, 2025",
category=AirflowProviderDeprecationWarning,
)
class TranslationLegacyModelTrainLink(BaseGoogleLink):
"""
Helper class for constructing Translation Legacy Model Train link.
Expand Down Expand Up @@ -163,6 +181,10 @@ def persist(
)


@deprecated(
planned_removal_date="September 30, 2025",
category=AirflowProviderDeprecationWarning,
)
class TranslationLegacyModelPredictLink(BaseGoogleLink):
"""
Helper class for constructing Translation Legacy Model Predict link.
Expand Down
Loading

0 comments on commit 9497e7c

Please sign in to comment.