update 2020-08-20 for preprints posted up until 2020-08-15

[bmkramer]

A couple of changes to the code:

• the arxiv script often is breaking off when querying the arxiv API, before processing all batches. Changing batch size from the default (100) to 1000 seemed to solve this. I also added a check on expected and returned results for arxiv.

• small code modification for including the institution field from Crossref (accommodating an apparent change in metadata structure)

• In ggplot2-3.3.4, ggsave can no longer be added to a ggplot addition-pipe (with +). Solved this by removing the +-sign as ggsave has a default input last_plot()

Also, still no new records (added weekly) were detected in RePEc since 2021-07-15. Need to keep an eye on this when running the next update.

[nicholasmfraser]

Hi @bmkramer, thanks for this. I tried to run through the update myself yesterday and found the same problems as you :) The arXiv issue is a strange one (also mentioned in ropensci/aRxiv#54). I experimented with the batch size but still got inconsistent numbers of results returned (also when I set the batch size to 1000). If this continues in future updates it may be necessary to change the source to the OAI-PMH interface. I was in the process of trying to figure out why I did not get any bioRxiv/medRxiv preprints included, but now I see this is due to the change in institution field from Crossref that you have fixed - thanks. Will merge these changes now.