Response from earthaccess devs about issues importing .tifs

[NobodyLivesHere]

So I had asked on the earthdata forum about the .tif import error that people were getting and it appears to indeed be an authentication issue

Here was the response

And here are the links they provided:

https://github.com/nasa/HLS-Data-Resources
https://github.com/nasa/HLS-Data-Resources/blob/main/python/tutorials/HLS_Tutorial.ipynb

I am currently trying to test this out, though my conda environment has been giving me issues with the gdal module.

[eggvoice]

I don't fully understand why my configurations are working yet but I figure I'd share in case it'd help you. I am on MacOS (Sonoma Version 14.2.1 (23C71)). Let us know what you end up doing.

Proof that it works.

Files that were created in my home directory:

ls -al | grep -E ".netrc|cookies.txt"
-rw-r--r--@   1 ed.chan  staff     79 Feb 17 22:39 .netrc
-rw-r--r--@   1 ed.chan  staff   2191 Feb 18 10:35 cookies.txt

I have a .env file in my repo (make sure you git ignore this one):

EARTHDATA_USERNAME=<your username>
EARTHDATA_PASSWORD=<your password>
GDAL_HTTP_MAX_RETRY=5
GDAL_HTTP_RETRY_DELAY=1
GDAL_HTTP_COOKIEFILE='~/cookies.txt'
GDAL_HTTP_COOKIEJAR='~/cookies.txt'
GDAL_DISABLE_READDIR_ON_OPEN='EMPTY_DIR'
CPL_VSIL_CURL_ALLOWED_EXTENSIONS='TIF'
GDAL_HTTP_UNSAFESSL='YES'

For that .netrc file, you could create by hand or by script:

Code snippet:

from dotenv import load_dotenv


username = os.getenv('EARTHDATA_USERNAME')
password = os.getenv('EARTHDATA_PASSWORD')

# earthaccess.login doesn't seem to be creating this file
netrc_fpath = os.path.expanduser("~/.netrc")
if not netrc_fpath:
    with open(netrc_fpath, 'w') as file:
        file.write(f"machine urs.earthdata.nasa.gov\nlogin {username}\npassword {password}\n")

Alternatively, you can just manually create that file:

machine urs.earthdata.nasa.gov
login <your_username>
password <your_password>

[eggvoice]

Forgot to say. You had to load the .env file too. Here's how:

from dotenv import load_dotenv

load_dotenv() # Load .env file

[eculler]

Nice work @eggvoice @NobodyLivesHere!

I wonder if it works for me because I have downloaded Earthdata files from a command line script in the past, and so probably still have the cookies set up?

@eggvoice if you log in using earthaccess.login it will do the .netrc file for you.

You can also use the os package to set environment variables in the code (although don’t out your password or username in plain text)

[eggvoice]

Thanks Elsa!

I moved all my environment variables to .env because I thought I needed to configure EARTHDATA_PASSWORD.

I was using strategy='environment' having read that

"environment": retrieve username and password from $EARTHDATA_USERNAME and $EARTHDATA_PASSWORD.
        persist (Boolean): will persist credentials in a .netrc file

It didn't create that .netrc file for whatever reasons so I created that via my snippet. Didn't really have a chance to further troubleshoot that one as I have moved on to the rest of the assignment.

[eggvoice]

Ye, it wasn't super clear to me what step in the process the cookies.txt was created. I also did some downloading via earthaccess.download() and perhaps that's when it was created.