fetchSDA_spatial: Unqueryable STATSGO mukeys -- water

[brownag]

Some individual mapunits in STATSGO (e.g. "Water (s8369)") exceed the JSON serialization limit when fetching gid and the full mupolygongeo. This is not terribly surprising, but I just wanted to post it since I ran into it with new functionality added to fetchSDA_spatial.

This is not fixable from the perspective of fetchSDA_spatial, and one currently needs to do some sort of a spatial window using SDA_spatialQuery to obtain target polygons. I am curious how many other nationally-extensive concepts we have like this? Probably none as pathological as "Water".

library(soilDB)
# requires latest (2.5.9+) soilDB off GH
bad <- fetchSDA_spatial(x = "657964", db = 'STATSGO', chunk.size = 1)
#> Using 1 chunks...
#> Bad chunk encountered. Querying each individually...
#> Warning: 
#> Internal server error: Error during serialization or deserialization using the JSON JavaScriptSerializer. The length of the string exceeds the value set on the maxJsonLength property.
#> Bad chunk encountered. Querying each individually...
#> Done in 39.5 secs; mean/chunk: 0 secs; mean/symbol: 39.47 secs.
#> Error in attr(s, "total.time") <- mintime: attempt to set an attribute on NULL

I discovered this when testing the STATSGO generalization of fetchSDA_spatial. Overall it works well, in the case below, only one mukey out of 78 in the "full" result exceeds maxJsonLength.

library(sp)
library(soilDB)

symbs <- c('CA077','CA632','CA644','CA630', 'CA628', 'CA649')

# get bounding boxes of survey areas
by.areasym.bbox <- fetchSDA_spatial(x = symbs, 
                                   by.col = "areasymbol",
                                   method = "bbox",
                                   geom.src = "sapolygon",
                                   add.fields = "legend.areaname")

# get overlapping STATSGO geometry
statsgo.overlap <- SDA_spatialQuery(by.areasym.bbox, 
                                    what = "geom", 
                                    db = "STATSGO")

# get unique mukey from overlap, and query their full extent
full.extent <- fetchSDA_spatial(x = unique(statsgo.overlap$mukey), 
                                db = 'STATSGO', chunk.size = 1,
                                add.fields = "legend.areaname")
# compare
par(mar=c(0,0,0,0))
plot(full.extent)
plot(statsgo.overlap, col="blue", add=T)
plot(by.areasym.bbox, lty=2, lwd=3, border="red", add=T)

Since the "areasymbol" for STATSGO is the "US" things like the below will be encountered when you do what I did above (can't show full detail geometry of course). This is "expected" behavior. Any reasonable prefiltering on mapunits to remove miscellaneous areas would fix this, but spatial queries (SDA_spatialQuery) are pretty likely to pick up these problematic MUKEYs.

library(sp)
#> Warning: package 'sp' was built under R version 4.0.3
library(soilDB)
bad.bbox <- fetchSDA_spatial(x = "657964", 
                             db = 'STATSGO', 
                             chunk.size = 1, 
                             method = "bbox",
                             add.fields = "mapunit.muname")
#> Using 1 chunks...
#> Chunk #1 completed (n = 1; 1.2 secs)
#> Done in 1.22 mins; mean/chunk: 73.4 secs; mean/symbol: 73.4 secs.
par(mar=c(0,0,0,0))
plot(bad.bbox, col="RED")

bad.bbox$muname[1]
#> [1] "Water (s8369)"

^{Created on 2020-12-01 by the reprex package (v0.3.0)}

[brownag]

OK, with soilDB @ cf91169 reprex 1 from above post now returns NULL rather than erroring out. That covers the "issue" here.

library(soilDB)
bad <- fetchSDA_spatial(x = "657964", db = 'STATSGO', chunk.size = 1)
#> Using 1 chunks...
#> Bad chunk encountered. Querying each individually...
#> Warning: 
#> Internal server error: Error during serialization or deserialization using the JSON JavaScriptSerializer. The length of the string exceeds the value set on the maxJsonLength property.
#> Bad chunk encountered. Querying each individually...
#> Warning in FUN(X[[i]], ...): Symbol 657964 dropped from result due to error! May
#> exceed the JSON serialization limit or have other topologic problems.
#> Done in 27.4 secs; mean/chunk: 0 secs; mean/symbol: 27.35 secs.
bad
#> NULL

^{Created on 2020-12-01 by the reprex package (v0.3.0)}