Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error crawling Letterboxd #3

Open
Donnnno opened this issue Apr 7, 2021 · 3 comments
Open

Error crawling Letterboxd #3

Donnnno opened this issue Apr 7, 2021 · 3 comments
Labels
bug Something isn't working

Comments

@Donnnno
Copy link

Donnnno commented Apr 7, 2021

I noticed that the crawler isn't able to crawl the letterboxd ToS and other policies anymore

The ToS:
https://letterboxd.com/legal/terms-of-use/
https://letterboxd.com/legal/privacy-policy/
https://letterboxd.com/legal/community-policy/

The edit page:
https://edit.tosdr.org/services/2074/annotate

& the Xpath:
/html/body/div[1]/div/div/article

It seems that our crawler wasn't able to retrieve any text. Please check that the XPath and URL are accurate.

Reason: TimeoutError
Stacktrace: Waiting for element to be located By(xpath, /html/body/div[1]/div/div/article) Wait timed out after 10002ms 
@JustinBack
Copy link
Member

Which crawler were you using?

That error message indicates the XPath is wrong as it cant find it in 10 seconds

@JustinBack
Copy link
Member

I just checked the documents and it seems you'd have to go with a full page crawling. Not exactly why this is happening but I copied the exact XPath as well and the page does not seem to alter it

@JustinBack JustinBack transferred this issue from tosdr/edit.tosdr.org Apr 8, 2021
@JustinBack JustinBack added the bug Something isn't working label Apr 8, 2021
@Donnnno
Copy link
Author

Donnnno commented Apr 8, 2021

Which crawler were you using?

The default tosdr page grawler.

I just checked the documents and it seems you'd have to go with a full page crawling.

Thanks for the quick reply, I'll do that in the meantime. Thanks :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants