Skip to content

Latest commit

 

History

History
88 lines (71 loc) · 3.63 KB

SECRETS.md

File metadata and controls

88 lines (71 loc) · 3.63 KB
Raw

Secrets

Operations in FarmVibes.AI can retrieve secrets to use as parameters, which can be useful to avoid storing secrets in plain-text. Secrets are stored safely within the Kubernetes cluster and are not transmited or visible outside the VM. For more information on how secrets within Kubernetes, refer to Kubernetes documentation.

Secrets may be added to the cluster through the add-secret command of the farmvibes-ai.sh script. The secret can then be passed as parameters to the workflow yaml files.

This document details how to add or delete a secret to the cluster, as well as lists all workflows that require a secret.

Adding a secret to FarmVibes.AI cluster

To add a secret with a key <key> and value <value>, run:

bash farmvibes-ai.sh add-secret <key> <value>

Using a secret within a workflow

Secrets are used in a workflow with the @SECRET notation. For example, @SECRET(my-keyvault-name, my-secret-key) in which my-secret-key is the key and my-keyvault-name is the key-vault. For local FarmVibes.AI instalation, the key-vault can be any non-empty string.

The following workflow yaml shows an example of an exposed secret parameter (download_password) with a default key (my-secret-pass):

name: my_test_wf
sources:
  input_a: 
    - download.input
sinks:
  output_b: download.output
parameters:
  download_password: "@SECRET(my-keyvault-name, my-secret-pass)"
tasks:
  download:
    op: my_exemple_op
    parameters:
      password: "@from(download_password)"
edges:
description:
  short_description:
    Example workflow.
  long_description:
    Requires secret from parameter download_password. 
    Default secret key is my-secret-pass.
  sources:
    input_a: Example input.
  sinks:
    output_b: Example output.
  parameters:
    download_password: Download password secret.

Deleting a secret to FarmVibes.AI cluster

bash farmvibes-ai.sh delete-secret <key>

List of workflows and their associated secrets

  • Ambient Weather API key (parameter api-key with default secret key ambient-api-key) and App key (parameter app-key with default secret key ambient-app-key).

    • data_ingestion/weather/get_ambient_weather

  • NOAA GFS SAS token (parameter noaa_gfs_token with default secret key noaa-gfs-sas).

    • data_ingestion/weather/get_forecast

  • SciHub username and password (parameters scihub_user and ``scihub_password, and default secret keys scihub-user` and `scihub-password`, respectively).

    • data_ingestion/sentinel1/preprocess_s1

  • Planetary computer API key. By default, FarmVibes.AI workflows access the Planetary Computer catalog anonymously, when possible. However, we recommend registering for an API key (see more information here) to avoid being throttled.

    • data_ingestion/dem/download_dem
    • data_ingestion/landsat/preprocess_landsat
    • data_ingestion/naip/download_naip
    • data_ingestion/sentinel1/preprocess_s1
    • data_ingestion/sentinel2/preprocess_s2
    • data_ingestion/sentinel2/preprocess_s2_improved_mask
    • data_ingestion/spaceeye/spaceeye
    • data_ingestion/spaceeye/spaceeye_interpolation
    • data_ingestion/spaceeye/spaceeye_preprocess
    • farm_ai/agriculture/canopy_cover
    • farm_ai/agriculture/change_detection
    • farm_ai/agriculture/emergence_summary
    • farm_ai/agriculture/methane_index
    • farm_ai/agriculture/ndvi_summary
    • farm_ai/agriculture/conservation_practices
    • farm_ai/agriculture/landsat_ndvi_trend
    • ml/dataset_generation/datagen_crop_segmentation