Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix scraping logic after site changes #8

Merged
merged 1 commit into from
Apr 21, 2024

Conversation

ndbroadbent
Copy link
Contributor

@ndbroadbent ndbroadbent commented Apr 20, 2024

It looks like the website has been updated, so I fixed the scraping logic.

Screenshot 2024-04-20 at 9 04 16 PM

Fixes #7

I'm running this change locally and it's working for me now:

image
❯ curl "http://10.5.7.10:32769/api/v1/rrext/?addr=MYADDRESS"
{"Collections":[{"Day":"Friday 26 April","Date":"2024-04-26T00:00:00+12:00","Rubbish":true,"Recycle":false,"FoodScraps":false},{"Day":"Friday 26 April","Date":"2024-04-26T00:00:00+12:00","Rubbish":false,"Recycle":false,"FoodScraps":true},{"Day":"Thursday 2 May","Date":"2024-05-02T00:00:00+12:00","Rubbish":false,"Recycle":true,"FoodScraps":false}],"Address":{"ACRateAccountKey":"KEY","Address":"MYADDRESS","Suggestion":"MYADDRESS"}}

~
❯ curl "http://10.5.7.10:32769/api/v1/rr/?addr=MYADDRESS"
{"rubbish":"2024-04-26","recycle":"2024-05-02","foodscraps":"2024-04-26","address":"MYADDRESS"}

@rusq
Copy link
Owner

rusq commented Apr 20, 2024

Hey @ndbroadbent, thank you!

@rusq
Copy link
Owner

rusq commented Apr 20, 2024

Looks like the test are upset, would you be able to have a look at them, otherwise I'll have a look tomorrow?

@rusq rusq changed the base branch from master to scraping-fix April 21, 2024 04:56
@rusq
Copy link
Owner

rusq commented Apr 21, 2024

Fixed the tests, but obviously can't push into your branch. Created a new branch, will merge it there, and then update the tests

@rusq rusq merged commit 926d584 into rusq:scraping-fix Apr 21, 2024
1 check failed
@rusq rusq mentioned this pull request Apr 21, 2024
@ndbroadbent
Copy link
Contributor Author

Oh right sorry about the tests! Thanks for the fix!

@rusq
Copy link
Owner

rusq commented Apr 21, 2024

Np, thanks for the PR!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Returns 0001-01-01 for /api/v1/rr, and empty Collections for /api/v1/rrext
2 participants