From af361de0bec8c40528498f55d4cf94f588122682 Mon Sep 17 00:00:00 2001 From: Pratichhya <39898768+Pratichhya@users.noreply.github.com> Date: Mon, 17 Jun 2024 15:45:45 +0200 Subject: [PATCH 1/3] inference using URL --- docs/machine_learning.rst | 26 ++++++++++++++++---------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/docs/machine_learning.rst b/docs/machine_learning.rst index 69f315e1b..47b59370d 100644 --- a/docs/machine_learning.rst +++ b/docs/machine_learning.rst @@ -89,7 +89,6 @@ Finally execute this whole training flow as a batch job:: training_job = model.create_job() training_job.start_and_wait() - Inference ---------- @@ -97,22 +96,29 @@ When the batch job finishes successfully, the trained model can then be used with the ``predict_random_forest`` process on the raster data cube (or another cube with the same band structure) to classify all the pixels. +Thus, the training job results is inspected to fetch the trained models url:: + + from openeo.rest.job import JobResults + + results : JobResults = training_job.get_results() + links = results.get_metadata()['links'] + ml_model_metadata_url = [link for link in links if 'ml_model_metadata.json' in link['href']][0]['href'] + print(ml_model_metadata_url) + + +Next, load the model from the URL:: + + from openeo.rest.mlmodel import MlModel + saved_model = MlModel.load_ml_model(connection=connection, id=ml_model_metadata_url) + Technically, the openEO ``predict_random_forest`` process has to be used as a reducer function inside a ``reduce_dimension`` call, but the openEO Python client library makes it a bit easier by providing a :py:meth:`~openeo.rest.datacube.DataCube.predict_random_forest` method directly on the :py:class:`~openeo.rest.datacube.DataCube` class, so that you can just do:: predicted = cube.predict_random_forest( - model=training_job.job_id, + model = saved_model, dimension="bands" ) predicted.download("predicted.GTiff") - - -We specified the model here by batch job id (string), -but it can also be specified in other ways: -as :py:class:`~openeo.rest.job.BatchJob` instance, -as URL to the corresponding STAC Item that implements the `ml-model` extension, -or as :py:class:`~openeo.rest.mlmodel.MlModel` instance (e.g. loaded through -:py:meth:`~openeo.rest.connection.Connection.load_ml_model`). From 023acc59c03f3aef792ccac67aa6cb557244afa6 Mon Sep 17 00:00:00 2001 From: Pratichhya <39898768+Pratichhya@users.noreply.github.com> Date: Fri, 21 Jun 2024 19:41:59 +0200 Subject: [PATCH 2/3] addressed suggestions --- docs/machine_learning.rst | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/docs/machine_learning.rst b/docs/machine_learning.rst index 47b59370d..83e5e29ac 100644 --- a/docs/machine_learning.rst +++ b/docs/machine_learning.rst @@ -98,9 +98,8 @@ with the ``predict_random_forest`` process on the raster data cube Thus, the training job results is inspected to fetch the trained models url:: - from openeo.rest.job import JobResults - results : JobResults = training_job.get_results() + results = training_job.get_results() links = results.get_metadata()['links'] ml_model_metadata_url = [link for link in links if 'ml_model_metadata.json' in link['href']][0]['href'] print(ml_model_metadata_url) @@ -108,8 +107,7 @@ Thus, the training job results is inspected to fetch the trained models url:: Next, load the model from the URL:: - from openeo.rest.mlmodel import MlModel - saved_model = MlModel.load_ml_model(connection=connection, id=ml_model_metadata_url) + model = connection.load_ml_model(id=ml_model_metadata_url) Technically, the openEO ``predict_random_forest`` process has to be used as a reducer function inside a ``reduce_dimension`` call, but the openEO Python client library makes it @@ -117,8 +115,16 @@ a bit easier by providing a :py:meth:`~openeo.rest.datacube.DataCube.predict_ran directly on the :py:class:`~openeo.rest.datacube.DataCube` class, so that you can just do:: predicted = cube.predict_random_forest( - model = saved_model, + model=saved_model, dimension="bands" ) predicted.download("predicted.GTiff") + +We specified the model here by URL corresponding to the +STAC Item that implements the ml-model extension, +but it can also be specified in other ways: +as :py:class:`~openeo.rest.job.BatchJob` instance, +as job_id of training job(string), +or as :py:class:`~openeo.rest.mlmodel.MlModel` instance (e.g. loaded through +:py:meth:`~openeo.rest.connection.Connection.load_ml_model`). \ No newline at end of file From c6a18fdd524c1af8571bc6c718f10e6b3e61eb5c Mon Sep 17 00:00:00 2001 From: Pratichhya <39898768+Pratichhya@users.noreply.github.com> Date: Mon, 24 Jun 2024 13:29:01 +0200 Subject: [PATCH 3/3] simple text updates --- docs/machine_learning.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/machine_learning.rst b/docs/machine_learning.rst index 83e5e29ac..40527922c 100644 --- a/docs/machine_learning.rst +++ b/docs/machine_learning.rst @@ -96,7 +96,7 @@ When the batch job finishes successfully, the trained model can then be used with the ``predict_random_forest`` process on the raster data cube (or another cube with the same band structure) to classify all the pixels. -Thus, the training job results is inspected to fetch the trained models url:: +We inspect the result metadata of the training job to obtain the STAC Item URL of the trained model:: results = training_job.get_results() @@ -115,7 +115,7 @@ a bit easier by providing a :py:meth:`~openeo.rest.datacube.DataCube.predict_ran directly on the :py:class:`~openeo.rest.datacube.DataCube` class, so that you can just do:: predicted = cube.predict_random_forest( - model=saved_model, + model=model, dimension="bands" ) @@ -125,6 +125,6 @@ We specified the model here by URL corresponding to the STAC Item that implements the ml-model extension, but it can also be specified in other ways: as :py:class:`~openeo.rest.job.BatchJob` instance, -as job_id of training job(string), +as job_id of training job (string), or as :py:class:`~openeo.rest.mlmodel.MlModel` instance (e.g. loaded through :py:meth:`~openeo.rest.connection.Connection.load_ml_model`). \ No newline at end of file