Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
This is a sync of our repo. Here's the summary of the changes: ## Support to new ADMA API version: **[💔 BREAKING CHANGES]** We added support to the new [Azure Data Manager for Agriculture (ADMA)](https://learn.microsoft.com/en-us/rest/api/data-manager-for-agri/?view=rest-data-manager-for-agri-dataplane-2023-11-01-preview) API version _2023-11-01-preview_. To comply with the new API, some of our data types, workflows, and notebooks were updated. A summary of the changes are: - The `farmer_id` property has now changed to `party_id` in [`ADMAgSeasonalFieldInput`](https://microsoft.github.io/farmvibes-ai/docfiles/code/vibe_core_data/farm.html#vibe_core.data.farm.ADMAgSeasonalFieldInput) object (which is passed as input for our workflows) and other [ADMA data types](https://microsoft.github.io/farmvibes-ai/docfiles/code/vibe_core_data/farm.html). Also, there is no more the `boundary_id` field as boundary resources are no more single objects in ADMA. The boundary information is part of the ADMA entities (seasonal field, field etc.). - All operations (seasonal field, field, prescriptions etc.) now have their own geometry. - Planting information in Seasonal Field was moved to a new operation ["Planting Data"](https://learn.microsoft.com/en-us/rest/api/data-manager-for-agri/dataplane/planting-data?view=rest-data-manager-for-agri-dataplane-2023-11-01-preview). - The affected workflows are: [`data_ingestion/admag/admag_seasonal_field`](https://microsoft.github.io/farmvibes-ai/docfiles/markdown/workflow_yaml/data_ingestion/admag/admag_seasonal_field.html), [`data_ingestion/admag/prescriptions`](https://microsoft.github.io/farmvibes-ai/docfiles/markdown/workflow_yaml/data_ingestion/admag/prescriptions.html), and [`farm_ai/agriculture/heatmap_using_classification_admag`](https://microsoft.github.io/farmvibes-ai/docfiles/markdown/workflow_yaml/farm_ai/agriculture/heatmap_using_classification_admag.html). - Likewise, the following notebooks have been updated: [notebooks/admag](https://github.com/microsoft/farmvibes-ai/tree/main/notebooks/admag) and [notebooks/heatmaps/nutrients_using_classification_admag](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/heatmaps/nutrients_using_classification_admag.ipynb). ## Workflows: - [🎉 NEW] We added new workflows to download and process forest extent data. These workflows include: (i) the Advanced Land Observing Satellite (ALOS) forest/non-forest map, (ii) the Global Land Analysis (GLAD) forest map, and (iii) the Global Forest Change (Hansen) datasets. The processing workflow detects forest changes using statistical tests (specifically, the Cochran-Armitage test) to determine whether the change represents a trend. - [📈 IMPROVEMENT] We are releasing an updated version of the SpaceEye workflow that leverages Sentinel-1 RTC products instead of GRD products with SNAP preprocessing. These changes lead to a significant speedup and a decrease in disk space consumption. Faster, lighter, and cloudless-er images to y'all! Due to this new version of the workflow, cached results are invalid and will be recomputed on new runs. - [📈 IMPROVEMENT] We extended the `data_ingestion/weather/download_herbie` workflow to allow downloading both analysis (zero lead time) up to a point in time and then forecast with progressively increasing lead times after that. - [⚒️ FIX] We added a [script](todo-add-link-to-script) to convert Segment Anything Model (SAM) weights to ONNX format and import it into the local FarmVibes.AI cluster to be used in the SAM workflows and notebooks. For instructions on how to use the script, refer to the _"Adding SAM's ONNX models to the cluster"_ entry in our [Troubleshooting page](https://microsoft.github.io/farmvibes-ai/docfiles/markdown/TROUBLESHOOTING.html). Related to issues #145 and #153. - [💔 **BREAKING CHANGES**] Planetary Computer API key is required for all workflows that download or use Sentinel-1 products (e.g., `data_ingestion/sentinel1/preprocess_s1` and `data_ingestion/spaceeye/spaceeye`). The documentation with the parameter description will be updated in a future release. ## Backend: - [📈 IMPROVEMENT] We updated our base images to use Python `3.11`, as well as DAPR `1.13.0`, fixing a few security vulnerabilities of the previous version that could lead to DoS in the kubernetes services. Additionally, our DAPR logs are now in JSON format. - [📈 IMPROVEMENT] We updated `ngrok` package to version `7.1.2`, which will solve the issues with the [What-if scenario evaluation for carbon sequestration](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/carbon/whatif.ipynb) notebook and `farm_ai/carbon_local/carbon_whatif` workflow. Related to issue #151. - [📈 IMPROVEMENT] We improved our worker retry policy and are now using exponential backoff for retries. This means workers will be less idle on long queues, avoiding cases where workers would wait for one minute before starting a queued task. - [⚒️ FIX] We fixed an issue with the `add-secret` subcommand not adding secrets to the KeyVault during remote (AKS) cluster setup. Related to issue #147. ## Notebooks: - [🎉 NEW] We added a few notebooks showcasing the new forest workflows, such as downloading [ALOS](todo-add-link-to-notebook) and [GLAD](todo-add-link-to-notebook) forest maps, and [evaluating forest changes over time with the Hansen maps](todo-add-link-to-notebook). We also provide a notebook highlighting how to detect increase/decrease trends in forest pixel levels over an user-input geometry and time range. ## Documentation: - [🎉 NEW] We added a new [REST API documentation page](https://microsoft.github.io/farmvibes-ai/docfiles/markdown/REST_API.html) describing the endpoints, parameters, and responses for our API. ## Client: - [📈 IMPROVEMENT & ⚒️ FIX] We made a few quality-of-life improvements and bug fixes to our python client. Users can now monitor the last `n` runs with `client.monitor(n)`, which will print a summarized table with their progress, as well as retrieve the last `n` `VibeWorkflowRun` objects with `client.get_last_runs(n)` method. We also added minor bug fixes to the `monitor` and `document_workflow` methods, which were breaking for deleated runs and workflows with None/empty value parameters, respectivelly. --------- Co-authored-by: Bruno Silva <[email protected]> Co-authored-by: Eduardo Rodrigues <[email protected]> Co-authored-by: Leonardo Nunes <[email protected]> Co-authored-by: Naga Bilwanth Gangarapu <[email protected]> Co-authored-by: Rafael Padilha <[email protected]> Co-authored-by: Renato Luiz de Freitas Cunha <[email protected]> Co-authored-by: Roberto de Moura Estevão Filho <[email protected]>
- Loading branch information
