New agents guide and deployment config docs updates

docs/deployment/agents/airflow.rst

@@ -0,0 +1,97 @@
+.. _deployment-agent-setup-airflow:
+
+Airflow agent
+=================
+
+This guide provides an overview of how to set up the Airflow agent in your Flyte deployment.
+Please note that the you don't need an Airflow cluster to run the Airflow tasks, since Flytekit will
+automatically compile Airflow tasks to Flyte tasks and execute them on the Flyte cluster.
+
+Specify agent configuration
+----------------------------
+
+.. tabs::
+
+    .. group-tab:: Flyte binary
+
+      Edit the relevant YAML file to specify the agent.
+
+      .. code-block:: bash
+
+        kubectl edit configmap flyte-sandbox-config -n flyte
+
+      .. code-block:: yaml
+        :emphasize-lines: 7,11,16
+
+        tasks:
+          task-plugins:
+            enabled-plugins:
+              - container
+              - sidecar
+              - k8s-array
+              - agent-service
+            default-for-task-types:
+              - container: container
+              - container_array: k8s-array
+              - airflow: agent-service
+
+        plugins:
+          agent-service:
+            supportedTaskTypes:
+            - airflow
+
+    .. group-tab:: Flyte core
+
+      Create a file named ``values-override.yaml`` and add the following configuration to it.
+
+      .. code-block:: yaml
+
+        configmap:
+          enabled_plugins:
+            # -- Tasks specific configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#GetConfig)
+            tasks:
+              # -- Plugins configuration, [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#TaskPluginConfig)
+              task-plugins:
+                # -- [Enabled Plugins](https://pkg.go.dev/github.com/flyteorg/flyteplugins/go/tasks/config#Config). Enable sagemaker*, athena if you install the backend
+                enabled-plugins:
+                  - container
+                  - sidecar
+                  - k8s-array
+                  - agent-service
+                default-for-task-types:
+                  container: container
+                  sidecar: sidecar
+                  container_array: k8s-array
+                  airflow: agent-service
+            plugins:
+              agent-service:
+                supportedTaskTypes:
+                - airflow
+
+
+Upgrade the Flyte Helm release
+------------------------------
+
+.. tabs::
+
+  .. group-tab:: Flyte binary
+
+    .. code-block:: bash
+
+      helm upgrade <RELEASE_NAME> flyteorg/flyte-binary -n <YOUR_NAMESPACE> --values <YOUR_YAML_FILE>
+
+    Replace ``<RELEASE_NAME>`` with the name of your release (e.g., ``flyte-backend``),
+    ``<YOUR_NAMESPACE>`` with the name of your namespace (e.g., ``flyte``),
+    and ``<YOUR_YAML_FILE>`` with the name of your YAML file.
+
+  .. group-tab:: Flyte core
+
+    .. code-block:: bash
+
+      helm upgrade <RELEASE_NAME> flyte/flyte-core -n <YOUR_NAMESPACE> --values values-override.yaml
+
+    Replace ``<RELEASE_NAME>`` with the name of your release (e.g., ``flyte``)
+
+    and ``<YOUR_NAMESPACE>`` with the name of your namespace (e.g., ``flyte``).
+
+For Airflow plugin on the Flyte cluster, see `Airflow Plugin Example <TODO: Add airflow plugin example>`_

docs/deployment/agents/bigquery.rst

@@ -1,6 +1,6 @@
 .. _deployment-agent-setup-bigquery:
 
-Google BigQuery Agent
+Google BigQuery agent
 ======================
 
 This guide provides an overview of setting up BigQuery agent in your Flyte deployment.
@@ -103,4 +103,4 @@ Upgrade the Flyte Helm release
 
     and ``<YOUR_NAMESPACE>`` with the name of your namespace (e.g., ``flyte``).
 
-For BigQuery plugin on the Flyte cluster, please refer to `BigQuery Plugin Example <https://docs.flyte.org/en/latest/flytesnacks/examples/bigquery_plugin/bigquery.html>`_
+For BigQuery plugin on the Flyte cluster, see `BigQuery Plugin Example <https://docs.flyte.org/en/latest/flytesnacks/examples/bigquery_plugin/bigquery.html>`_

docs/deployment/agents/databricks.rst

@@ -1,6 +1,6 @@
 .. _deployment-agent-setup-databricks:
 
-Databricks Agent
+Databricks agent
 =================
 
 This guide provides an overview of how to set up Databricks agent in your Flyte deployment.
@@ -291,4 +291,4 @@ Wait for the upgrade to complete. You can check the status of the deployment pod
 
   kubectl get pods -n flyte
 
-For databricks plugin on the Flyte cluster, please refer to `Databricks Plugin Example <https://docs.flyte.org/en/latest/flytesnacks/examples/databricks_plugin/index.html>`_
+For databricks plugin on the Flyte cluster, see `Databricks Plugin Example <https://docs.flyte.org/en/latest/flytesnacks/examples/databricks_plugin/index.html>`_

docs/deployment/agents/index.md

@@ -2,31 +2,40 @@
 
 # Agent Setup
 
-.. tags:: Agent, Integration, Data, Advanced
+```{tags} Agent, Integration, Data, Advanced
+```
+
+To set configure your Flyte deployment for agents, see the documentation below.
 
-Discover the process of setting up Agents for Flyte.
+:::{note}
+If you are using a managed deployment of Flyte, you will need to contact your deployment administrator to configure agents in your deployment.
+:::
 
 ```{list-table}
 :header-rows: 0
 :widths: 20 30
 
-* - {ref}`Bigquery Agent <deployment-agent-setup-bigquery>`
-  - Guide to setting up the Bigquery agent.
+* - {ref}`Airflow Agent <deployment-agent-setup-airflow>`
+  - Configuring your Flyte deployment for the Airflow agent
+* - {ref}`Databricks Agent <deployment-agent-setup-databricks>`
+  - Configuring your Flyte deployment for the Databricks agent.
+* - {ref}`Google BigQuery Agent <deployment-agent-setup-bigquery>`
+  - Configuring your Flyte deployment for the BigQuery agent.
 * - {ref}`MMCloud Agent <deployment-agent-setup-mmcloud>`
-  - Guide to setting up the MMCloud agent.
+  - Configuring your Flyte deployment for the MMCloud agent.
 * - {ref}`Sensor Agent <deployment-agent-setup-sensor>`
-  - Guide to setting up the Sensor agent.
-* - {ref}`Databricks Agent <deployment-agent-setup-databricks>`
-  - Guide to setting up the Databricks agent.
+  - Configuring your Flyte deployment for the sensor agent.
 ```
 
 ```{toctree}
 :maxdepth: 1
 :name: Agent setup
 :hidden:
 
+airflow
+databricks
 bigquery
 mmcloud
-databricks
 sensor
+snowflake
 ```

docs/deployment/agents/mmcloud.rst

@@ -118,4 +118,4 @@ Wait for the upgrade to complete. You can check the status of the deployment pod
 
   kubectl get pods -n flyte
 
-For MMCloud plugin on the Flyte cluster, please refer to `Memory Machine Cloud Plugin Example <https://docs.flyte.org/en/latest/flytesnacks/examples/mmcloud_plugin/index.html>`_
+For MMCloud plugin on the Flyte cluster, see `Memory Machine Cloud Plugin Example <https://docs.flyte.org/en/latest/flytesnacks/examples/mmcloud_plugin/index.html>`_

docs/deployment/agents/sensor.rst

@@ -1,13 +1,13 @@
 .. _deployment-agent-setup-sensor:
 
-Sensor Agent
+Sensor agent
 =================
 
-Sensor enables users to continuously check for a file or a condition to be met periodically.
+The sensor agent enables users to continuously check for a file or a condition to be met periodically.
 
 When the condition is met, the sensor will complete.
 
-This guide provides an overview of how to set up Sensor in your Flyte deployment.
+This guide provides an overview of how to set up the sensor agent in your Flyte deployment.
 
 Spin up a cluster
 -----------------
@@ -43,7 +43,7 @@ Spin up a cluster
 Specify agent configuration
 ----------------------------
 
-Enable the Sensor agent by adding the following config to the relevant YAML file(s):
+Enable the sensor agent by adding the following config to the relevant YAML file(s):
 
 .. tabs::
 
@@ -77,7 +77,7 @@ Enable the Sensor agent by adding the following config to the relevant YAML file
 
     .. group-tab:: Flyte core
 
-      Create a file named ``values-override.yaml`` and add the following configuration to it.
+      Create a file named ``values-override.yaml`` and add the following configuration to it:
 
       .. code-block:: yaml
 

docs/deployment/agents/snowflake.rst

@@ -0,0 +1,102 @@
+.. _deployment-agent-setup-snowflake:
+
+Snowflake agent
+=================
+
+This guide provides an overview of how to set up the Snowflake agent in your Flyte deployment.
+
+1. Set up the key pair authentication in Snowflake. For more details, see the `Snowflake key-pair authentication and key-pair rotation guide <https://docs.snowflake.com/en/user-guide/key-pair-auth>`__.
+2. Create a secret with the group "snowflake" and the key "private_key". For more details, see `"Using Secrets in a Task" <https://https://docs.flyte.org/en/latest/flytesnacks/examples/productionizing/use_secrets.html>`__.
+
+.. code-block:: bash
+   kubectl create secret generic snowflake-private-key --namespace=flytesnacks-development --from-file=your_private_key_above
+
+Specify agent configuration
+----------------------------
+
+.. tabs::
+
+    .. group-tab:: Flyte binary
+
+      Edit the relevant YAML file to specify the agent.
+
+      .. code-block:: bash
+
+        kubectl edit configmap flyte-sandbox-config -n flyte
+
+      .. code-block:: yaml
+        :emphasize-lines: 7,11,16
+
+        tasks:
+          task-plugins:
+            enabled-plugins:
+              - container
+              - sidecar
+              - k8s-array
+              - agent-service
+            default-for-task-types:
+              - container: container
+              - container_array: k8s-array
+              - snowflake: agent-service
+
+        plugins:
+          agent-service:
+            supportedTaskTypes:
+            - snowflake
+
+    .. group-tab:: Flyte core
+
+      Create a file named ``values-override.yaml`` and add the following configuration to it.
+
+      .. code-block:: yaml
+
+        configmap:
+          enabled_plugins:
+            # -- Tasks specific configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#GetConfig)
+            tasks:
+              # -- Plugins configuration, [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#TaskPluginConfig)
+              task-plugins:
+                # -- [Enabled Plugins](https://pkg.go.dev/github.com/flyteorg/flyteplugins/go/tasks/config#Config). Enable sagemaker*, athena if you install the backend
+                enabled-plugins:
+                  - container
+                  - sidecar
+                  - k8s-array
+                  - agent-service
+                default-for-task-types:
+                  container: container
+                  sidecar: sidecar
+                  container_array: k8s-array
+                  snowflake: agent-service
+            plugins:
+              agent-service:
+                supportedTaskTypes:
+                - snowflake
+
+Ensure that the propeller has the correct service account for BigQuery.
+
+Upgrade the Flyte Helm release
+------------------------------
+
+.. tabs::
+
+  .. group-tab:: Flyte binary
+
+    .. code-block:: bash
+
+      helm upgrade <RELEASE_NAME> flyteorg/flyte-binary -n <YOUR_NAMESPACE> --values <YOUR_YAML_FILE>
+
+    Replace ``<RELEASE_NAME>`` with the name of your release (e.g., ``flyte-backend``),
+    ``<YOUR_NAMESPACE>`` with the name of your namespace (e.g., ``flyte``),
+    and ``<YOUR_YAML_FILE>`` with the name of your YAML file.
+
+  .. group-tab:: Flyte core
+
+    .. code-block:: bash
+
+      helm upgrade <RELEASE_NAME> flyte/flyte-core -n <YOUR_NAMESPACE> --values values-override.yaml
+
+    Replace ``<RELEASE_NAME>`` with the name of your release (e.g., ``flyte``)
+
+    and ``<YOUR_NAMESPACE>`` with the name of your namespace (e.g., ``flyte``).
+
+For Snowflake plugin on the Flyte cluster, see `Snowflake Plugin Example <https://docs.flyte.org/en/latest/flytesnacks/examples/snowflake_plugin/snowflake.html>`_

docs/flyte_agents/configuring_your_flyte_deployment_for_agents.md

@@ -0,0 +1,16 @@
+---
+jupytext:
+  formats: md:myst
+  text_representation:
+    extension: .md
+    format_name: myst
+---
+
+(configuring_deployment_for_agents)=
+# Configuring your Flyte deployment for agents
+
+To configure your Flyte deployment for a particular agent, see the [Agent Setup](https://docs.flyte.org/en/latest/deployment/agents/index.html) documentation.
+
+:::{note}
+If you are using a managed deployment of Flyte, you will need to contact your deployment administrator to configure agents in your deployment.
+:::