Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upload asset to GEE cloud project; is it possible? #339

Open
3 tasks done
RetoSchmucki opened this issue Jun 19, 2023 · 2 comments
Open
3 tasks done

Upload asset to GEE cloud project; is it possible? #339

RetoSchmucki opened this issue Jun 19, 2023 · 2 comments

Comments

@RetoSchmucki
Copy link

  • rgee version: 1.1.5
  • R version: 4.3.0
  • Operating System: Windows

At submit an issue, please attached the following information of your rgee session:

  • You have the Python API installed (from terminal):
earthengine -h
  • You can find the credentials file on your system:
library(rgee)
ee_path <- path.expand("~/.config/earthengine/credentials")
file.exists(ee_path)
  • You can run a simple EE command from R:
library(rgee)

# Initialize the Earth Engine module.
ee_Initialize()

# Print metadata for a DEM dataset.
print(ee$Image('USGS/SRTMGL1_003')$getInfo())

python: C:/Users/xxxx/AppData/Local/Continuum/miniconda3/envs/rgee/python.exe
libpython: C:/Users/xxxx/AppData/Local/Continuum/miniconda3/envs/rgee/python38.dll
pythonhome: C:/Users/xxxx/AppData/Local/Continuum/miniconda3/envs/rgee
version: 3.8.17 | packaged by conda-forge | (default, Jun 16 2023, 07:01:59) [MSC v.1929 64 bit (AMD64)]
Architecture: 64bit
numpy: C:/Users/xxxx/AppData/Local/Continuum/miniconda3/envs/rgee/Lib/site-packages/numpy
numpy_version: 1.24.3
ee: C:\Users\xxxx\AppData\Local\CONTIN-1\MINICO-1\envs\rgee\lib\site-packages\ee_init_.p

NOTE: Python version was forced by RETICULATE_PYTHON

Description

More a question than an Issue. Can we upload assets to GEE cloud project? Everything works perfectly fine when uploading GeoTiff to my gee user/USERNAME/ using the raster_as_ee() function. But the same does not work with my GEE Cloud Project that has ``projects/ee-PROJECTNAME/assets/` as root path.

Interestingly, I can read ee_manage_assetlist() for my GEE Cloud Project, returning the correct path projects/ee-PROJECTNAME/assets/ but then when using raster_as_ee(), everything run smoothly, uploading the GeoTiff on GCS bucket and then

Running the OS command: earthengine upload image --manifest ...

But then, when trying to move it to GEE project I encounter some issue wth invalid root path.

State: FAILED
ERROR in Earth Engine servers: Not an assetserver id: 'projects/ee-PROJECTNAME/assets/planet'

I can't find any example working with a GEE project and cloud assets, only user/USERNAME/assets. Is this simply because the earthenging API can't write into cloud assets?

I searched and read the GEE documentation referring to the asset paths for users and projects, and everything is okay in the manifest name.

Did someone manage to upload assets to project cloud assets? Could it be related to my SaK permission?

What I Did

      myData <- fromJSON(file = file.path(dir_list[j], rast_metadata_lst[i]))

        ## The specific metadata are found in myData
        # myData$mosaic$bbox
        # myData$mosaic$first_acquired
        # myData$mosaic$last_acquired
        # myData$mosaic$coordinate_system
        # myData$mosaic$grid$resolution
        # myData$quad$id
        # myData$mosaic$name
        # myData$quad$percent_covered

        ## get the GeoTiff to upload
        r_ <- read_stars(file.path(dir_list[j], paste0(myData$quad$id, "_quad.tif")))

        ## build the asset name whit quad_id, year and month
        assetID <- paste0(
            sprintf("%s/%s",  ee_get_assethome(), "quad_"),
            myData$quad$id, "_",
            lubridate::year(myData$mosaic$first_acquired), "_",
            lubridate::month(myData$mosaic$first_acquired)
        )

        ## upload to GEE, using the GCS bucket "planet_gee" and add properties start and end times
        ee_stars_02 <- raster_as_ee(
            x = r_,
            bucket = "planet_gee",
            overwrite = TRUE,
            assetId = assetID,
            start_time = lubridate::as_date(myData$mosaic$first_acquired),
            end_time = lubridate::as_date(myData$mosaic$last_acquired) - 1
        )
@RetoSchmucki
Copy link
Author

maybe part of the answer is here:
https://developers.google.com/earth-engine/guides/image_manifest#asset-ids-and-names

For assets in Cloud projects, use projects/my_cloud_project/assets/my_asset.

For older legacy projects, the asset name in the manifest needs to be slightly different from the asset ID visible elsewhere in Earth Engine. To upload assets whose asset IDs start with users/some_user or projects/some_project, the asset name in the manifest must have the string projects/earthengine-legacy/assets/ prepended to the ID. For example, EE asset ID users/username/my_geotiff should be uploaded using the name projects/earthengine-legacy/assets/users/username/my_geotiff.

@RetoSchmucki
Copy link
Author

Update

I just tried to upload the asset (image) with the earthengine CLI, writing the manifest.json and omitting the startprojects/earthengine-legacy/assets/ from the asset name and only use projects/my_cloud_project/assets/my_asset and that works fine; uploading the asset in my Cloud project assets.

Something that could be added to the raster_as_ee() with an argument CloudProject = TRUE.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant