Empty response from Core Reporting API inside a Time Partitioned Query breaks the loop

[shahkushan1]

When extracting unsampled metrics for dimensions like PagePath for a relatively longer date range, the API returns a null response if no data is available for that date. This breaks the loop.

[igbuch]

Any progress on this issue @shahkushan1?

Thanks

[BobbyBarbeau]

@igbuch The workaround I've used is to simply modify GetDataFeed.R locally to:

GetDataFeed <- function(query.uri) {

  GA.Data <- GET(query.uri)
  GA.list <- ParseDataFeedJSON(GA.Data)
  if (is.null(GA.list$rows)) {
    cat("Your query matched 0 results. Please verify your query using the Query Feed Explorer and re-run it.\n")
    # break
    return(GA.list)
  } else {
    return (GA.list)
  }
}

The original if statement is:

  if (is.null(GA.list$rows)) {
    cat("Your query matched 0 results. Please verify your query using the Query Feed Explorer and re-run it.\n")
    break
  } else {
    return (GA.list)
  }
}

By commenting out the break and returning GA.list as NULL even when GA.list$rows is NULL, the error message is still output to the console, but the daywise API queries continue, correctly incrementing to the next day and so on. For my purposes, everything works as desired.

I don't know that this solution is scalable, because there are certainly instances where you wouldn't want to continue hammering the GA API with a daywise query that is fundamentally flawed. In that case, you'd would want the user to verify the query.

But for my purposes, I know the query is fine because it's one I regularly use. There just happens to be no matching data on a particular day.

This hack lets everything keep moving, but it absolutely does cut out the safety net that keeps you from bombarding the GA API with a query that will never return results and so should be used with some caution.

[JIsernhagen]

Would someone please be kind enough to suggest the equivalent local workaround for GetReportData (instead of GetDataFeed)? I am trying to use GetReportData, and am having exactly the same problem as is described above with null sets, but I don't understand the function above well enough to adapt it to create a version of GetDataFeed that presses on through "Your query matches 0 results..."

Thank you very much for your assistance.

[jdeboer]

When a query returns a response with zero results, perhaps an empty data.frame (with the appropriate columns (names and data types) but with no rows) could be returned. This way all the responses can be bound together as rows, as they will all have matching data.frame columns.

In the rga package I'm not sure but I think this is being handled here: https://github.com/skardhamar/rga/blob/master/R/core.R#L163

In the ganalytics package, this situation is handled as described above and can be seen in the following code:
https://github.com/jdeboer/ganalytics/blob/master/R/GaGetCoreReport.R#L138

Hopefully something can be applied here in this package too, probably somewhere in this file I'm guessing: https://github.com/Tatvic/RGoogleAnalytics/blob/master/R/GetDataFeed.R#L16

[dikesh]

Hi @JIsernhagen @jdeboer,
We are planning to release the updated version with all the bug fixes soon. Please give us sometime. :)

[JIsernhagen]

Hi Dikesh: are you able to be a little more precise? It would be worth the price of a twelve-pack to me if you implemented the empty set fix this week.

[JIsernhagen]

jdeboer: Can you advise me how to write an outside-the-function error handler which moves on to the next loop element when the "Error in GetDataFeed(query.uri) : no loop for break/next, jumping to top level" message is received? I have read what documentation I can find on try-error and tryCatch and can make no sense of it. All I want is "on-error next"-type functionality in this one, specific case.

Thanks for any guidance you can provide.

[jdeboer]

I'm not familiar with the error you are describing as I haven't attempted this myself yet. However, if you want to write an exception hander in R, I would certainly recommend the tryCatch function.

# Try `expr` and catch it if it throws an error, then do something else.
tryCatch(
  expr = {
    # insert here the code you want to try.
    thisFunctionDoesNotExist(1, 2, 3) # An example of something that will throw an error.
  },
  error = function(e) {
    # Leave this empty if you do not want anything to happen when there is an error,
    # i.e. the tryCatch will finish and the next command in R will be executed.
    # Otherwise, put in the code you want to run if `expr` throws an error.
    # You can get information about the error from the object `e`.
    message("Ha ha, an error occurred, oh well, who cares?")
    message(paste("In case you do care, the error was:", e))
  }
  # There are other optional arguments too, such as `warning` if you want to handle warnings.
)
# OK, now let's move on to the next thing...
message("Hello world!")

I hope the example I have written above is helpful. You can copy and paste it into the R console to see what happens.

Good luck!

[JIsernhagen]

I actually got it to work reasonably well using "Try." So thank you for your help. I appreciate your responsiveness.

[TomasKoerse]

@JIsernhagen Would you mind posting your script? I'm having a lot of trouble using TryCatch. Would be very much appreciated!

[JIsernhagen]

The subroutine is legally the property of the company I work for, but here is a relevant subset that should show how "try" can be applied:

if(inherits(p, "try-error")){
  print("Empty set, moving to two-day crawler")
  #first set
  query.list <- Init(start.date = date.start, end.date = date.start.5, dimensions = pulls$dimension_string[3], metrics = pulls$metric_string[3], max.results = 10000, sort = pulls$sort_string[3], table.id = ga_properties$ga_id[j])
  ga.query <- QueryBuilder(query.list) # Create Query Builder object so that the query parameters are validated
  p <- try(GetReportData(ga.query, token, split_daywise = T))
  if(inherits(p, "try-error")){
    print("First crawl set failed.")
    #next
  } else {
    p3.ass<-rbind(p, p3.ass)
  } 
  #second set
  query.list <- Init(start.date = date.start.4, end.date = date.start.3, dimensions = pulls$dimension_string[3], metrics = pulls$metric_string[3], max.results = 10000, sort = pulls$sort_string[3], table.id = ga_properties$ga_id[j])
  ga.query <- QueryBuilder(query.list) # Create Query Builder object so that the query parameters are validated
  p <- try(GetReportData(ga.query, token, split_daywise = T))
  if(inherits(p, "try-error")){
    print("Second crawl set failed.")
    #next
  } else {
    p3.ass<-rbind(p, p3.ass)
  } 
  #third set
  query.list <- Init(start.date = date.start.2, end.date = date.end, dimensions = pulls$dimension_string[3], metrics = pulls$metric_string[3], max.results = 10000, sort = pulls$sort_string[3], table.id = ga_properties$ga_id[j])
  ga.query <- QueryBuilder(query.list) # Create Query Builder object so that the query parameters are validated
  p <- try(GetReportData(ga.query, token, split_daywise = T))
  if(inherits(p, "try-error")){
    print("Third crawl set failed.")
    #next
  } else {
    p3.ass<-rbind(p, p3.ass)
  } 
} else {
  p3.ass<-rbind(p, p3.ass)
}