CloudFlare protected providers

[elgatito]

There was built-in CloudFlare bypass in Elementum, which helped to solve CloudFlare defense and get pages.
But with the latest defense update we cannot bypass it, so providers, that use CloudFlare to avoid robots (like us) to request pages, are not accessible for the moment.

I don't know any easy way of solving this issue.
Would be great if someone could help on this.

[iz8mbw]

Hi @elgatito Denis, hi all.
I'm not a developer so I cannot help on this, but what about to use FlareSolverr https://github.com/FlareSolverr/FlareSolverr ?
Thanks!

[elgatito]

@iz8mbw It requires installation of Nodejs on running host.
If you think it can help you - you can install it manually and use in Burst, as a proxy server.

[iz8mbw]

@elgatito Denis I use "Jackett" with "FlareSolverr" on my Linux ARM64 board.
I don't have ideas how to configure Burst to use my "Jackett" (listening on HTTP on my LAN) for searches.

[iz8mbw]

@elgatito I mean if it's possible to set to Burst to use "Jackett" please let me know how to do that.

[iz8mbw]

@elgatito I know there is this addon https://github.com/fugkco/script.elementum.jackett but it's not compatible with Kodi 19 (Python 3) so I cannot use and test it.
If it's possible to set Burst to use "Jackett" please let me know how to do that.
Many thanks!

[elgatito]

@iz8mbw I have no experience with Jackett.
#246

[iz8mbw]

@elgatito I have added "Jackett.json" into my "userdata/addon_data/script.elementum.burst/providers/" directory but I cannot see this custom provider into Elementum Burst list of providers both Public and Private.
Where I wrong?

[elgatito]

@iz8mbw You cannot see custom providers in settings. They are supposed to be enabled by default.

[iz8mbw]

OK, thanks!

[elgatito]

@iz8mbw @hoshsadiq
I can add, if someone makes a Kodi plugin which runs/installs FlareSolverr - then Burst could even check for this addon and proxy requests through it.

Same with Jackett. When separate addon is complete - I can add it to Elementum repository, so that it gets available when you install Elementum.

[hoshsadiq]

Did you mean to tag me? Think you meant @fugkco.

Anyway, haven't used Elementum in a while, seems FlareSolverr is using hcaptcha-solver, however, it doesn't seem to be working:

At this time none of the captcha solvers work. You can check the status in the open issues. Any help is welcome.

So I'm not really sure what the point would be to add FlareSolverr support until there's a solid method of solving the captcha.

[iz8mbw]

There was built-in CloudFlare bypass in Elementum, which helped to solve CloudFlare defense and get pages.
But with the latest defense update we cannot bypass it, so providers, that use CloudFlare to avoid robots (like us) to request pages, are not accessible for the moment.

I don't know any easy way of solving this issue.
Would be great if someone could help on this.

Hi @elgatito Denis.
May be this https://github.com/jairoxyz/script.module.cloudscraper can help?

[elgatito]

@iz8mbw Requires JavaScript engine.

[ShlomiD83]

https://github.com/NoahCardoza/CloudProxy

@elgatito how about this project?

[elgatito]

@ShlomiD83 Have no idea. If it works for you - you can use it as proxy in Burst.

[ShlomiD83]

I wish I knew how to use it, don't have a clue.

[burekas7]

@elgatito

There is an addon that called "mando".
I see there are some intersting functions that maybe could help,
I don't know actually, I guess you will know if so better than me.

The two functions are:
cloudflare_request (resources/modules/general.py)
get_tokens_with_headers (resources/modules/cfscrape/run.py) => Maybe other files in the "cfscrape" folder.

You can download the addon from here:
https://f2h.io/1co1sn73arhf

[burekas7]

https://github.com/NoahCardoza/CloudProxy

@elgatito how about this project?

It requires to create and use a server based on javascript/nodejs,
So this is not a proper solution.

[elgatito]

Any development help is welcome.
I have no time for this.

[fugkco]

The Jackett plugin is pretty much beta. Has been working for me, but I know there's a few bugs that people have reported that I've not really been able to look into. Plus no Kodi Matrix support, which I know Kodi is gearing up for.

If Elementum repo gets the latest releases from the GitHub releases API, I say feel free to add it to the repository, but if not, maybe wait a little bit longer until I've had a chance to migrate to 19 and get a minute to migrate the addon too.

[antonsoroko]

probably makes sense to pin this issue.

[elgatito]

@antonsoroko It is already pinned for 2 years. That is the only pinned issue, actually, here.

[elgatito]

@burekas7 @SubEdict @antonsoroko

What providers are now not accessible because of Cloudflare? Apart of Idope.

[antonsoroko]

@elgatito
ilcorsaroblu Torrentleech iptorrents - those do not work for me and for other people as I can see in other issues. In browser I literally see cloudflare page first and then redirect. Same page in burst debug output.

New yggtorrent URL also has Cloudflare but works for me for some reason (I made a PR for it).
Also have it (at least in headers, but I do not see the page and redirect), but work:
iTorrent
YTS
limetorrents

[SubEdict]

@elgatito I'm using it right now.
All English Public providers works (excluding #341).

[elgatito]

@antonsoroko

New yggtorrent URL also has Cloudflare but works for me for some reason (I made a PR for it).

They have different levels of protection set on different websites. Some check basic stuff and we usually pass that check. Some have a javascript page that needs browser interaction. Some require checking checkbox with a browser. Some require solving a captcha.

On sites like ilcorsaroblu if we want to download torrent - Cloudflare allows to do it. If trying to login to website - they require to pass a checkbox test.

They also detect a platform, so that same Burst/same Elementum, same settings, but running on another platform can lead to a Cloudflare check failure.

[antonsoroko]

Just out of interest I also tried to add my cookies (after passing the check) to burst cookie file, but without success.
Anyway, for me those "protected" trackers are not interesting, not sure how popular they are.

[elgatito]

@antonsoroko It will not work like that with a cookie, because it checks for all the headers and signatures when you use the cookie.

@scakemyer had a simple, but good solution - https://github.com/scakemyer/cloudhole .

Basically, you installed a plugin into a browser, and that plugin uploads your headers and Cloudflare cookies into a website. Later Burst was taking those cookies and was able to fetch those pages.

Why it worked:

• You get Cloudflare cookies for a specific website, and even if there was a captcha and you solved it - this check would be passed.
• Since you request it from same network where Burst/Elementum works - you will not have problems reusing cookies.
• Since you have all the headers, used to get Cloudflare cookies - you will not have problems.

But that extension was for Firefox. If someone wants to make a plugin for Chrome - I can make a server part, that would store those secrets in the same way cloudhole is storing them.

[burekas7]

@burekas7 @SubEdict @antonsoroko

What providers are now not accessible because of Cloudflare? Apart of Idope.

1337x starts using it :/
I tried to find any mirror that doesn't use it and the search is correct, but there isn't.

Do we have any solution for these prolematic CloudFlate providers by the way?

[ShlomiD83]

@burekas7 @SubEdict @antonsoroko
What providers are now not accessible because of Cloudflare? Apart of Idope.

1337x starts using it :/ I tried to find any mirror that doesn't use it and the search is correct, but there isn't.

Do we have any solution for these prolematic CloudFlate providers by the way?

you can try to bypass it using Jackett, FlareSolverr & script.elementum.jackett.
it seems to return results in Jackettt, need to test it inside Kodi.

[elgatito]

@ShlomiD83 @burekas7 @SubEdict #414 is now in testing.

[elgatito]

I managed to make ilcorsaroblu/torrentleech to work with Burst 0.0.87 and cookie sync enabled for those domains and itorrents.org.

One of them started to work when I changed Burst Useragent to become like the browser that used to create those cookies (otherwise Cloudflare was giving robot protection page).

By default Cookie Sync does not store user agent and TLS fingerprints.

[antonsoroko]

i did some testing:

torrentgalaxy (own protection) - previously it was giving me captcha page in burst debug log, with "cookie sync" - ok

cloudflare:
extranet (ext.to and extranet.torrentbay.st) - fail, still gives "Just a moment" page
idope - fail, still gives "Just a moment" page
yggtorrent - fail, still gives "Just a moment" page. but here it looks like extension does not add cf_clearance cookie to gist. if i manually add my cf_clearance to .kodi/temp/burst/common_cookies.jar - site works.

(tried even with/without my own "user agent" - same result)

other providers do not give me "Just a moment" page, so can't say anything about them.

---

would be nice if other people will try this feature as well.