Merging BSP into rpscrape data

[rmwesley99]

Hi,
Just wondering if anyone had tried merging Betfair Starting Price (BSP) into the data scraped from the Racing Post website by rpscrape?
I have used the Betfair data download manager to get BSP data for the last couple of years and was just planning to merge it into my RP dataset based on matching a) horse name, b) course name and c) date. Has anyone tried this already and found any gotchas, or a simpler way to do it?
Cheers, Richard

[rmwesley99]

Well, I've done it using a DataFrame merge. It is relatively simple to just merge on lowercase horse name and date without any issues. Interestingly some of my unprofitable back tests became profitable when marked against BSP rather than ISP, commission included. The profitable ones became more profitable.

[joenano]

I have a joining script that I use to add Betfair data, but it's not public. There were some issues with name changes of horses moving to Hong Kong and Australia iirc. it's linked to my own version of rpscrape which is now quite different from the public version but I will see what I can do.

[rmwesley99]

Hi @joenano ...no worries if you are busy since I have got something working, however always interested to see what others are doing if you do have time 👍 I wrote a wget script to download 10+ years worth of the daily BSP files from [https://promo.betfair.com/betfairsp/prices] then collated them into a single CSV / pandas DataFrame. I was then able to get an almost 100% match between the RP data and the BF data by merging on a) horse name (converted in both datasets to lowercase & stripping out the region from the RP horse name) and b) date. In the few instances where a BSP price isn't found, I just populate the starting price using ISP from the RP data as a backup. Cheers.