Login Issue

[gr8er1mark]

Hi All

I am having an issue with TGTG login. Previously I would receive the TGTG email and open the link on a device without the app and everything works as expected. However since installing the last update this no longer seems to work. Clicking the TGTG link opens the web browser and show the standard message that sign is has been successful but then I continue to get the emails asking me to login.

When I check the add ons log it shows the below error which makes it looks like a captcha is stopping the add on login.

TgtgLoginError(response.status_code, response.content)
tgtg.exceptions.TgtgLoginError: (403, b'{"url":"https://geo.captcha-delivery.com/captcha/

Appreciate any help you might be able to offer.

Thanks

[thetux07]

hello same problem here

[liudger]

turn of the add-on. (wait 30 min to be sure) reboot and you should receive an email when you have boot on start up enabled for the add-on.

[LJ-Stevo]

Still having trouble keeping it up and running...
In this log I started fresh at 23/12/24 at 18:00 by waiting more then a day to be unblocked again. (somewhere in the middel of the txt file)
Fresh installed, so I get the mail to log in again. Then it works... for 4 hours... Again Captcha... Suddenly it works again?? Without doing anything? But then back again Captcha. Have fun reading my log :-)
I hope you can find a fix for this, because I loved it realy much!
f1c878cb_tgtg-ha-mqtt-bridge_2024-12-24T19-45-47.607Z.log

[LJ-Stevo]

Still having trouble keeping it up and running... In this log I started fresh at 23/12/24 at 18:00 by waiting more then a day to be unblocked again. (somewhere in the middel of the txt file) Fresh installed, so I get the mail to log in again. Then it works... for 4 hours... Again Captcha... Suddenly it works again?? Without doing anything? But then back again Captcha. Have fun reading my log :-) I hope you can find a fix for this, because I loved it realy much! f1c878cb_tgtg-ha-mqtt-bridge_2024-12-24T19-45-47.607Z.log

Little update: If I check regularly the app itself, from time to time I first get a screen that checks something like the captcha but I can't remember what it says. I'm gonna try to take a screenshot next time.
But since then, the error is gone. if the captcha error returns, I just have to go to the app and that check screen appears again, and all is good for a while.
But there must be a solution for this, because that can't be forever like this... Now I always keep an eye on it, but that is not the intention of this add-on ofcourse! ;-)
So please help me (or us) out.

[LJ-Stevo]

If the captcha error returns, I just have to go to the app and that check screen appears again, and all is good for a while.

When I'm at home, I can't do that outside without having my home IP adress!
@MaxWinterstein I can give you an extra "coffee" if you can fix this! ;-)

[HanBarn]

I got the 403 error too.
It looks like TGTG has changed something (again..) so the IP is blocked faster than before.
My configuration is already changed. The polling is now every 15 minutes during the daytime (08:00-20:00).
This means a not frequent polling interval.
This should be no problem but I am getting the 403 error too and the Captcha.
Stopping the Add-on for some time is a temporary fix.
I don’t think it is related to the Add-on but more related to TGTG itself.
The TGTG app however does not have any issues and is running fine.
(using the same IP, Wifi network)

[MaxWinterstein]

I removed some code that stored a cookie by datadome, if I remember it correctly. I thought this was already handled within the upstream library, or superfluous at least.

I will have a second look at this, it might be an good explanation for it. Maybe I wrote something about that cookie in my traffic analysis with mitm proxy I did back then.

Fingers crossed I find some time these days 🤞

[MaxWinterstein]

It seems like the datadome cookie thing was in fact merged in the upstream tgtg repo (seems like I was involved in it, too 😄 ahivert/tgtg-python@ee19450 )

It seems like datadome, the company used by tgtg to prevent bad guys doing bad things on their API, got more restrictive I guess.

There are multiple related issues out there:

• Captcha error ahivert/tgtg-python#321
• Captcha - you have been blocked Chouffy/home_assistant_tgtg#129
• and some more

I had some little play with the API, and ended in the well known

tgtg.exceptions.TgtgAPIError: (429, 'Too many requests. Try again later.')

Which also blocked the whole Android APP on my phone as well.

---

I might need to wait a little bit to get unblocked again, or nuke half my network to get a new IP, and have a fresh look at the traffic between the app and and API.

Tricky situation.

[MaxWinterstein]

Small update:

I digged around with mitmproxy and looked at the requests of the iPhone app. It also got hit immediately with the datadome captcha thing.

After solving, the token and datadome cookie seem to be 'transportable', at least a small

curl -H 'content-type: application/json' -H 'accept: application/json' \
-H 'authorization: Bearer <myBearer>' --compressed \
-H 'user-agent: TooGoodToGo/24.12.0 (34781) (iPhone/iPhone 8; iOS 16.7.10; Scale/2.00/iOS)' \
-H 'cookie: datadome=<myCookie>' -X POST https://apptoogoodtogo.com/api/item/v8/18214 \
-d '{"origin":{"longitude":<myLong>,"latitude":<myLat>}}'

worked as expected.

I am not that deep into captcha solving / bypassing, but this seems nice to read - even though its made by a company selling 'bybassing' stuff: https://www.zenrows.com/blog/datadome-bypass#reverse-engineering-antibots
and also https://github.com/javapuppteernodejs/solving-datadome-captcha

As my local dev version works fine, even with a token and cookie that was some days old, and finely refreshed the token, I guess having a valid one might work for a few days - or more.

What might work: Enhance the login with some interactive captcha solving. Solving it manually is fine I guess. Maybe this can be wrapped with some selenium magic, I an not really into that kind of stuff.
When opening the captcha page, and solving it, an ajax request is sent to datadome, that gets a valid cookie as response. Quick googling did not help me to get that response back into my code programmatically. 😞

[LJ-Stevo]

A lot is just Chinese for me, but I have faith in you! :-)

[Shusaku1]

Hi everyone,

I'm new here and trying to catch up with the current situation regarding the TGTG API and login challenges.

So far, I’ve attempted the following steps:

Tried using the API directly, but it wouldn’t allow me to log in, redirecting me to a CAPTCHA page instead.
To bypass this, I set up an emulator running a rooted Android device with Magisk. This allowed me to install a Mitmproxy CA certificate, enabling me to monitor HTTP/HTTPS traffic but not for TGTG. The app detects something is wrong and I can't do any action.
My goal was to intercept the authentication tokens needed for logging in, but it seems that just installing the Mitmproxy CA certificate isn’t sufficient (maybe there is certificate pinning.)

I wanted to ask:
Have you encountered similar issues with certificate pinning while working with the API or monitoring TGTG traffic?
Is Mitmproxy a step you also went through, or were you able to use the API without these additional steps?
I’d appreciate any advice! If there’s a workaround or a preferred method
Thanks!

[wenzet]

Before finding this repository I implemented something similar with the same python package. Overall the implementation was more or less the same - just way simpler.

Back then I had similar issues when fetching too many shops too fast. So I added a delay between each get_item() call.
It was a bit of trial and error but 15 seconds appeared to work more or less.
@MaxWinterstein Maybe it's worth to have a look into #226. It's more like a draft but maybe that could work - at least as some type of workaround.

[HanBarn]

Before finding this repository I implemented something similar with the same python package. Overall the implementation was more or less the same - just way simpler.

Back then I had similar issues when fetching too many shops too fast. So I added a delay between each get_item() call. It was a bit of trial and error but 15 seconds appeared to work more or less. @MaxWinterstein Maybe it's worth to have a look into #226. It's more like a draft but maybe that could work - at least as some type of workaround.

Is there any progress on this issue?
Is it possible to add a time delay manually somewhere as a workaround?
TGTG is still temporary blocking me even if the polling interval is 5 minutes or more.
May be less favorite shops is an option too to limit the amount of requests.

[LJ-Stevo]

The only way I can keep it "working" is to open the app on the phone when I'm home, so then I have the same IP address.
When I get the captcha error in the log of tgtg add-on, I just go to the app and I instant get a verifying screen. When thats done, the next polling is working again for a few hours. But the point is ofcourse, checking the app all the time is just that we want to avoid by using this add-on. So I'm also hoping there will be a work around for this...

Edit:
Is this something that can help?
https://habr.com/en/articles/865274/

[HanBarn]

The only way I can keep it "working" is to open the app on the phone when I'm home, so then I have the same IP address. When I get the captcha error in the log of tgtg add-on, I just go to the app and I instant get a verifying screen. When thats done, the next polling is working again for a few hours. But the point is ofcourse, checking the app all the time is just that we want to avoid by using this add-on. So I'm also hoping there will be a work around for this...

Edit: Is this something that can help? https://habr.com/en/articles/865274/

I have never seen something in the TGTG app on my iPhone when the captcha message is showing in the log.
The app on my iPhone is working fine and I can refresh the info as much as needed.
Of course HA and my iPhone are on the same network at home.

[LJ-Stevo]

[LJ-Stevo]

@HanBarn
The app on my iPhone is working fine and I can refresh the info as much as needed.

I was refreshing in my favorite list so much because I'm waiting for one to come up and suddenly I get a screen to check if I'm human. So refreshing a lot in the app does that too for me. And I mean a lot, like for 5 min non stop...

[HanBarn]

@HanBarn The app on my iPhone is working fine and I can refresh the info as much as needed.

I was refreshing in my favorite list so much because I'm waiting for one to come up and suddenly I get a screen to check if I'm human. So refreshing a lot in the app does that too for me. And I mean a lot, like for 5 min non stop...

Allright, that makes sense if you use the TGTG a lot to check if there will be a box available for you.
Fact is that the app does allow quite a lot of requests.
Far more than we now see using the integration.
I hope there will be a solution soon so we can use the integration again as before.

[LJ-Stevo]

New problem I think...

I restarted and this is what I get:

404.txt

[HanBarn]

New problem I think...

I restarted and this is what I get:

404.txt

Yes.
There are more issues related to this.
marklagendijk/node-toogoodtogo-watcher#258

[HanBarn]

It looks like there is a fix in the TGTG API client.
Max, can you please checks this?
@MaxWinterstein

See this:
ahivert/tgtg-python#336

[MaxWinterstein]

It looks like there is a fix in the TGTG API client. Max, can you please checks this? @MaxWinterstein

See this: ahivert/tgtg-python#336

will have a deeper look later on, but at first peek it seems not to fix the login flow stuff, just the refresh endpoint for existing ones

[MaxWinterstein]

I just found a few minutes for this project, and I am somehow confused: it works?

Can someone verify this for me? Is it still broken for everyone?

[wenzet]

@MaxWinterstein
It does - but after some time it stops working. Got it working today after reinstalling and just had a look couple of minutes ago: It stopped after almost exactly 4 hrs. Then 404 again.

[MaxWinterstein]

@wenzet this should be fixed with the release I just did a few moments ago (just here, TGTG Addon needs to be released, hope to get this done today)

The main dependency tgtg was updated to 0.18.2 which had some fixes regarding the refresh endpoint, see ahivert/tgtg-python#336

Those 4 hours are hardcoded here:

https://github.com/ahivert/tgtg-python/blob/3fb879a87c9034b6a0dd94339bf85c087ed2560c/tgtg/__init__.py#L33

[MaxWinterstein]

The god of bad scripts was with me and the release is ready 🥳

Updating worked flawless for me, did not need to uninstall before:

Still confused what happened, and why the weird issues are gone, but I take it.