Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to crawl a page with encoding other than utf8 #17

Open
yogiatpozen opened this issue Jan 21, 2016 · 0 comments
Open

Unable to crawl a page with encoding other than utf8 #17

yogiatpozen opened this issue Jan 21, 2016 · 0 comments

Comments

@yogiatpozen
Copy link

Hi,

I'm trying to crawl a page (like http://021-online.com) which has charset=gb2312.
When I try to read head/title field using cheerio (that's embedded) I got, let say, crap instead of proper chars.

Am I missing some configuration, or is it a bug that prevents from properly crawling non-utf8 pages?
It may also be a problem of cheerio rather than roboto itself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant