Post new error when web service is down (instead of file already exists message).

[mahdin75]

Hello,

I'm running the following code, and It doesn't work for me:

library(prism)
prism_set_dl_dir("E:/prism")
get_prism_annual("ppt", years = 1999, keepZip = FALSE)

I get this message:
already exists. Skipping downloading.

Operating System: Microsoft Windows 10
R version: 4.1.0
Prism version: 0.2.1

[sheilasaia]

I used this package in early November without any trouble, but recently ran into this same issue while using the get_prism_dailys() function, although, I later tried other get_prism… functions using the example code and am getting similar results.

I made sure to set my download directory and then requested data as I’ve done in the past. However, while pulling data that was not already in my archive, I got a message print out telling me that the data “already exists. Skipping downloading.” This was an unexpected message since I was requesting data that I had not requested before. I even tried to run the code as it’s written in the example (i.e., get_prism_dailys(type = "ppt", dates = "2013/06/01", keepZip = FALSE)) but this was giving me the same issue and I did not already have it downloaded. To look into this more, I used httr::GET("http://services.nacse.org/prism/data/public/4km") and the output returned a 403 error (Forbidden). When I went to the PRISM site, I noticed that the ftp site (https://ftp.prism.oregonstate.edu/) was working fine and I could access the data there using my web browser.

I’m not sure where the best place to put this would be, but I wanted to suggest to add a http status check to the prism_webservice() function for when the OSU PRISM group website is up (200 status) or down (403, etc.). That way you could print out a message to let the user will know that the issue is on the server side (i.e., the OSU PRISM group’s server is down) and is not related to the user’s computer/archive functioning.

Thank you for all you’ve done to develop this very helpful package! 🙌

I'm on a Windows 11 machine. R version is 4.3.1 and prism R package version is 0.2.1.

[rabutler-usbr]

It does look like the web service is down. I'll reach out to them to see (this happened before).

And thank you, that will be a great improvement. I'll update the code to provide a different error message when GET() returns 403, instead of the message that the file has already been downloaded.

Thank you!

[rabutler-usbr]

> tmp <- httr::GET("http://services.nacse.org/prism/data/public/4km")
> dput(tmp)
structure(list(url = "https://services.nacse.org/prism/data/public/4km", 
    status_code = 403L, headers = structure(list(date = "Tue, 28 Nov 2023 18:37:09 GMT", 
        server = "Apache", `content-length` = "199", `content-type` = "text/html; charset=iso-8859-1"), class = c("insensitive", 
    "list")), all_headers = list(list(status = 301L, version = "HTTP/1.1", 
        headers = structure(list(date = "Tue, 28 Nov 2023 18:37:09 GMT", 
            server = "Apache", location = "https://services.nacse.org/prism/data/public/4km", 
            `content-length` = "256", `content-type` = "text/html; charset=iso-8859-1"), class = c("insensitive", 
        "list"))), list(status = 403L, version = "HTTP/1.1", 
        headers = structure(list(date = "Tue, 28 Nov 2023 18:37:09 GMT", 
            server = "Apache", `content-length` = "199", `content-type` = "text/html; charset=iso-8859-1"), class = c("insensitive", 
        "list")))), cookies = structure(list(domain = logical(0), 
        flag = logical(0), path = logical(0), secure = logical(0), 
        expiration = structure(numeric(0), class = c("POSIXct", 
        "POSIXt")), name = logical(0), value = logical(0)), row.names = integer(0), class = "data.frame"), 
    content = as.raw(c(0x3c, 0x21, 0x44, 0x4f, 0x43, 0x54, 0x59, 
    0x50, 0x45, 0x20, 0x48, 0x54, 0x4d, 0x4c, 0x20, 0x50, 0x55, 
    0x42, 0x4c, 0x49, 0x43, 0x20, 0x22, 0x2d, 0x2f, 0x2f, 0x49, 
    0x45, 0x54, 0x46, 0x2f, 0x2f, 0x44, 0x54, 0x44, 0x20, 0x48, 
    0x54, 0x4d, 0x4c, 0x20, 0x32, 0x2e, 0x30, 0x2f, 0x2f, 0x45, 
    0x4e, 0x22, 0x3e, 0x0a, 0x3c, 0x68, 0x74, 0x6d, 0x6c, 0x3e, 
    0x3c, 0x68, 0x65, 0x61, 0x64, 0x3e, 0x0a, 0x3c, 0x74, 0x69, 
    0x74, 0x6c, 0x65, 0x3e, 0x34, 0x30, 0x33, 0x20, 0x46, 0x6f, 
    0x72, 0x62, 0x69, 0x64, 0x64, 0x65, 0x6e, 0x3c, 0x2f, 0x74, 
    0x69, 0x74, 0x6c, 0x65, 0x3e, 0x0a, 0x3c, 0x2f, 0x68, 0x65, 
    0x61, 0x64, 0x3e, 0x3c, 0x62, 0x6f, 0x64, 0x79, 0x3e, 0x0a, 
    0x3c, 0x68, 0x31, 0x3e, 0x46, 0x6f, 0x72, 0x62, 0x69, 0x64, 
    0x64, 0x65, 0x6e, 0x3c, 0x2f, 0x68, 0x31, 0x3e, 0x0a, 0x3c, 
    0x70, 0x3e, 0x59, 0x6f, 0x75, 0x20, 0x64, 0x6f, 0x6e, 0x27, 
    0x74, 0x20, 0x68, 0x61, 0x76, 0x65, 0x20, 0x70, 0x65, 0x72, 
    0x6d, 0x69, 0x73, 0x73, 0x69, 0x6f, 0x6e, 0x20, 0x74, 0x6f, 
    0x20, 0x61, 0x63, 0x63, 0x65, 0x73, 0x73, 0x20, 0x74, 0x68, 
    0x69, 0x73, 0x20, 0x72, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 
    0x65, 0x2e, 0x3c, 0x2f, 0x70, 0x3e, 0x0a, 0x3c, 0x2f, 0x62, 
    0x6f, 0x64, 0x79, 0x3e, 0x3c, 0x2f, 0x68, 0x74, 0x6d, 0x6c, 
    0x3e, 0x0a)), date = structure(1701196629, class = c("POSIXct", 
    "POSIXt"), tzone = "GMT"), times = c(redirect = 0.10472, 
    namelookup = 0.004287, connect = 0.132216, pretransfer = 0.175762, 
    starttransfer = 0.2593, total = 0.25938), request = structure(list(
        method = "GET", url = "http://services.nacse.org/prism/data/public/4km", 
        headers = c(Accept = "application/json, text/xml, application/xml, */*"), 
        fields = NULL, options = list(useragent = "libcurl/8.3.0 r-curl/5.1.0 httr/1.4.5", 
            httpget = TRUE), auth_token = NULL, output = structure(list(), class = c("write_memory", 
        "write_function"))), class = "request"), handle = <pointer: 0x0000014e8cebd3f0>), class = "response")

[rabutler-usbr]

With above commit, the following error message is now produced if status is not 200:

Warning messages:
1: In prism_webservice(uris_post81[i], keepZip) : 
Attempted to download:
http://services.nacse.org/prism/data/public/4km/ppt/1999
Could not reach url, with status:
403
Please try pasting one or more of the following urls into your web browser:
https://services.nacse.org/prism/data/public/4km/tmin/20090405
https://services.nacse.org/prism/data/public/4km/tmin/200904
https://services.nacse.org/prism/data/public/4km/tmin/2009
https://services.nacse.org/prism/data/public/4km/tmin/1944
If you receive the same status code, then the issue is with the PRISM web service.
Please try again later or use the manual downloads at https://prism.oregonstate.edu.
If the status is different or download works, please file a bug at:
https://github.com/ropensci/prism/issues/

---

Have to wait until the web service is back up to test and make sure fix didn't affect things when web service is working.

Thank you for the great idea @sheilasaia

[rabutler-usbr]

Web service is back up

[mahdin75]

Thank you very much. It's working now.

[sheilasaia]

That warning message is perfect, @rabutler-usbr! 🙌

Very glad that the web service is back up. 🎉 Thank you again for this awesome R package.