Better avoidance of 403/429

[dmytrokoren]

Modified webdriver and updated requirements

[jdholtz]

Do you have a Docker image on your repository for this? I think it’d be good to check that this fixes #279 before merging it.

[dmytrokoren]

Do you have a Docker image on your repository for this? I think it’d be good to check that this fixes #279 before merging it.

Yes I have updated :develop tag with these new changes.

[dmytrokoren]

@jdholtz can you run :develop tag and see if it works for you, I have made new touch ups. I'll be making new PR or merge to existing!?

[jdholtz]

@jdholtz can you run :develop tag and see if it works for you, I have made new touch ups. I'll be making new PR or merge to existing!?

Yes, it's working for me. You can add the new changes here. So it looks like the two parameters (including this change and #274) are using incognito and is_mobile when initializing the driver. Were you able to reproduce the 403/429 and the incognito parameter fixed it?

[dmytrokoren]

@jdholtz can you run :develop tag and see if it works for you, I have made new touch ups. I'll be making new PR or merge to existing!?

Yes, it's working for me. You can add the new changes here. So it looks like the two parameters (including this change and #274) are using incognito and is_mobile when initializing the driver. Were you able to reproduce the 403/429 and the incognito parameter fixed it?

Yes, I was running it on Mac docker image and I was getting 403 on first try but then I added incognito and the 403 went away. Might be coincidence but I think it still did the trick.

[dmytrokoren]

@jdholtz can you run :develop tag and see if it works for you, I have made new touch ups. I'll be making new PR or merge to existing!?

Yes, it's working for me. You can add the new changes here. So it looks like the two parameters (including this change and #274) are using incognito and is_mobile when initializing the driver. Were you able to reproduce the 403/429 and the incognito parameter fixed it?

Intermittently I do see 403/429 when running multiple times (while testing). Today in docker I'm running 1 account and its running every 4 hours, I did not see any 429 or 403.

[dmytrokoren]

This PR is related to #279

[puluceno]

any plans to get this merged? 😄

[jdholtz]

any plans to get this merged? 😄

Yes but changes related to 403/429s are still being worked on and discussed in #277 and #279.

[Bubba8291]

@dmytrokoren what is the status on this? I have a check in on Tuesday, and I can use this script and post the logs like last time.

[dmytrokoren]

@dmytrokoren what is the status on this? I have a check in on Tuesday, and I can use this script and post the logs like last time.

#It's up to date, with pending PR's. 429 error still coming on login at times. But 403 for me is kind of faded.

[Bubba8291]

@dmytrokoren what is the status on this? I have a check in on Tuesday, and I can use this script and post the logs like last time.

#It's up to date, with pending PR's. 429 error still coming on login at times. But 403 for me is kind of faded.

Wait is the issues just for login? I've always used this script by configuring it for reservations, not accounts.

[dmytrokoren]

@dmytrokoren what is the status on this? I have a check in on Tuesday, and I can use this script and post the logs like last time.

#It's up to date, with pending PR's. 429 error still coming on login at times. But 403 for me is kind of faded.

Wait is the issues just for login? I've always used this script by configuring it for reservations, not accounts.

For me it was mostly login now. If you use reservations you should be fine, most likely will get seats lower B's depends where you flying.
When I did checking from Dallas to NYC I got B15... from NyC to Dallas I got A35, which was awesome

[Bubba8291]

@dmytrokoren what is the status on this? I have a check in on Tuesday, and I can use this script and post the logs like last time.

#It's up to date, with pending PR's. 429 error still coming on login at times. But 403 for me is kind of faded.

Wait is the issues just for login? I've always used this script by configuring it for reservations, not accounts.

For me it was mostly login now. If you use reservations you should be fine, most likely will get seats lower B's depends where you flying. When I did checking from Dallas to NYC I got B15 which was awesome but from NyC to Dallas I got A35

So you're saying using the account method gives you a better chance at getting a better boarding position? If so, why is that?

[dmytrokoren]

@dmytrokoren what is the status on this? I have a check in on Tuesday, and I can use this script and post the logs like last time.

#It's up to date, with pending PR's. 429 error still coming on login at times. But 403 for me is kind of faded.

Wait is the issues just for login? I've always used this script by configuring it for reservations, not accounts.

For me it was mostly login now. If you use reservations you should be fine, most likely will get seats lower B's depends where you flying. When I did checking from Dallas to NYC I got B15 which was awesome but from NyC to Dallas I got A35

So you're saying using the account method gives you a better chance at getting a better boarding position? If so, why is that?

No reservation and login makes no difference. I was just stating the fact that there is still good possibility to get good seats using this script :) depending where you flying to

[Bubba8291]

@dmytrokoren what is the status on this? I have a check in on Tuesday, and I can use this script and post the logs like last time.

#It's up to date, with pending PR's. 429 error still coming on login at times. But 403 for me is kind of faded.

Wait is the issues just for login? I've always used this script by configuring it for reservations, not accounts.

For me it was mostly login now. If you use reservations you should be fine, most likely will get seats lower B's depends where you flying. When I did checking from Dallas to NYC I got B15 which was awesome but from NyC to Dallas I got A35

So you're saying using the account method gives you a better chance at getting a better boarding position? If so, why is that?

No reservation and login makes no difference. I was just stating the fact that there is still good possibility to get good seats using this script :) depending where you flying to

Ah. Yep I feel pretty confident as long as the script gets a 200 on the first request. I will post the verbose once I run it in a few days :)

[jdholtz]

@dmytrokoren is this ready to be reviewed again or are the 403/429 mitigations still being worked on?

[dmytrokoren]

@jdholtz lets review this PR?

[ne0ark]

Still running into following issue:

2024-09-19 18:02:00 DEBUG Process-1[webdriver:253]: login_request_id set successfully
2024-09-19 18:02:00 DEBUG Process-1[webdriver:405]: Stopped virtual display successfully
2024-09-19 18:02:00 DEBUG Process-1[webdriver:311]: Logging in failed for an unknown reason
2024-09-19 18:02:00 DEBUG Process-1[reservation_monitor:223]: Encountered a Too Many Requests error while logging in. Retrying
2024-09-19 18:02:00 DEBUG Process-1[webdriver:396]: Started virtual display successfully
2024-09-19 18:02:00 DEBUG Process-1[webdriver:158]: Starting webdriver for current session
2024-09-19 18:02:01 DEBUG Process-1[webdriver:172]: Using browser version: 119.0.6045.159
2024-09-19 18:02:01 DEBUG Process-1[webdriver:198]: Retrieving new headers
2024-09-19 18:02:01 DEBUG Process-1[webdriver:178]: Loading Southwest check-in page (this may take a moment)
2024-09-19 18:02:08 DEBUG Process-1[webdriver:112]: Logging into account to get a list of reservations and valid headers
Notice: Encountered a Too Many Requests error while logging in for Jdove. Skipping reservation retrieval until next interval``` 

I wonder if the time for checks needs to be truly randomized.

[dmytrokoren]

If you are using Raspberry Pi there is no dedicated ARM chromium driver so I believe undetected chromium will not work and you will be detected.

[dmytrokoren]

@jdholtz @ne0ark lets review this PR. This is final change working fine for me. I think Raspberry Pi will have issues but that's because there is no dedicated ARM chromium as I read.

[jdholtz]

@dmytrokoren, would it be possible to split this PR into a few different ones? There's 3 major changes happening (cached headers, display in Docker, and retries for failed logins). I'd prefer if these changes could be split up so there aren't as many review comments/changes to make (or time to review them) and each feature can be merged when it's ready rather than needing to wait for all three.

[jdholtz]

I think Raspberry Pi will have issues but that's because there is no dedicated ARM chromium as I read.

Chromium is installed in Ubuntu ARM versions, which should be sufficient for our purposes. There have been people running this script on ARM architecture without having issues that the Pi is having. Not sure why though...

[Bubba8291]

@dmytrokoren What is the most recent build from your PRs? I got another check in this week and I wanna make sure for when I test it, it's using yours.

[Bubba8291]

@dmytrokoren Nevermined. The hurricane made our plans change

[dmytrokoren]

@dmytrokoren Nevermined. The hurricane made our plans change

Stay safe

[dmytrokoren]

@jdholtz, the current develop branch is working great, but I still occasionally get 429 Too Many Requests errors on login. I'll raise a new pull request to add a retry mechanism to address that. We can hold off on the cached headers for now, as fetching new headers each time is working fine. Let me know how soon you can review new PR?

[ne0ark]

Hello, there is a memory leak somewhere. I am using dmytrokoren/auto-southwest-check-in:develop and my memory usage spiked to 6 gig.

[Bubba8291]

@dmytrokoren I am getting ready to configure the script again. Is new_dev the branch I should be using for the latest commits?