-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(sso): respect X-Forwarded-*
headers for redirect URL. Fixes #13031
#13609
base: main
Are you sure you want to change the base?
Conversation
X-Forwarded-*
headers for redirect URL. Fixes #13031
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We are having the exact same issue. Thank you for coding this up.
This adds a new `make start UI_SECURE=true` flag that can be used to test Argo with a TLS termination proxy in front, which is needed for testing argoproj#13609. Also, `make start SECURE=true UI=true` was broken because `ui/src/app/webpack.config.js` wasn't respecting the `ARGO_SECURE` flag, so I fixed that too. Note that `make start UI_SECURE=true PROFILE=sso` doesn't work without manually changing the `redirectUrl` here: https://github.com/argoproj/argo-workflows/blob/29ea3518a30b2b605290d0ba38f95bfb75a2901f/manifests/quick-start/sso/overlays/workflow-controller-configmap.yaml#L15 I tried to see if I could automate this, but I couldn't figure out a clean way of doing so. I guess I could create a new profile called `sso-secure` under `test/e2e/manifests/`? Signed-off-by: Mason Malone <[email protected]>
…roj#13031 When running Argo in plain text mode behind a TLS termination proxy with SSO enabled, you will always be redirected to the workflows page after logging in, regardless of which page you came from. This is happening because `server/auth/sso/sso.go` is using the protocol and host of the request to determine the redirect URL, which can be different than the original request when a proxy is used. `oauth2-proxy` had a similar issue, which they fixed in oauth2-proxy/oauth2-proxy#957 by reading from the [X-Forwarded-Host](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-Host) and [X-Forwarded-Proto](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-Proto) headers, if set. This applies the same fix. The only major difference is that `oauth2-proxy` only reads those headers when the `--reverse-proxy` option is set, but I didn't add a corresponding option here. The only reason I can think of for doing so is to preserve backwards-compatibility, since it's theoretically possible this could break someone's setup. I can't think of any real-world situations where that can happen, but let me know if I should put this behind a option and I'm happy to do so. Tested using `make start UI_SECURE=true PROFILE=sso` and changing this line: https://github.com/argoproj/argo-workflows/blob/29ea3518a30b2b605290d0ba38f95bfb75a2901f/manifests/quick-start/sso/overlays/workflow-controller-configmap.yaml#L15 to `redirectUrl: https://localhost:8080/oauth2/callback`. I tried to see if I could automate this, but I couldn't figure out a clean way of doing so. Signed-off-by: Mason Malone <[email protected]>
FYI: I split off the dev environment improvements to #13674 |
proto := "http" | ||
|
||
if r.URL.Scheme != "" { | ||
// Get protocol from the X-Forwarded-Proto header if set to support running Argo behind a TLS termination proxy | ||
if r.Header.Get("X-Forwarded-Proto") != "" { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
my main concern with changing configuration based on proxy is that it becomes susceptible to MITMs or similar -- anyone can send such a header.
Whereas redirectURL
is configured by an admin
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm this was previously using the request URL (r.URL.Scheme
), which I believe could be similarly spoofed 🤔 .
HandleCallback
wasn't using that though, but was using r.Host
, although that's technically not exactly the same as the host header in Go (related #12691)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, but that was already an inherent problem with this setup, because if you're running a TLS termination in front of Argo, you have to trust everything in between you and the proxy (including the proxy). The only way of protecting against a malicious proxy is through E2E encryption.
Also, if someone does MITM the connection and has the ability to inject arbitrary headers, then they almost certainly have the ability to completely rewrite the response body too, in which case there's no need for them to redirect the user to a malicious page. They can just directly serve up whatever content they want,.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With the specific setup of the issue, yes, but this potentially extends it further too. Even with E2EE, a maliciously crafted request could potentially send someone to an uncontrolled/malicious URL, similar to an open redirect.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Even with E2EE, a maliciously crafted request could potentially send someone to an uncontrolled/malicious URL, similar to an open redirect.
I can't think of any attack vector where that can happen that wasn't already a viable attack vector. If E2E encryption is used, the only means for an attacker to inject arbitrary headers is to either compromise the Argo server or the end user's device, and in both cases the attacker isn't going to bother with open redirect vulnerabilities because they can just rewrite the page to be whatever they want.
Regardless, I entered #13747 with an alternative approach: use relative redirect URLs. That sidesteps the whole issue.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If E2E encryption is used, the only means for an attacker to inject arbitrary headers is to either compromise the Argo server or the end user's device
The simplest and one of the most common variants is to, for instance, write an informative blog post that sends requests to the API, and one of those requests is maliciously crafted. The user then copy+pastes it unknowingly.
In CVSS, this kind of exploit requires "User Interaction", but does not require compromising the user's device
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, I'm still not seeing how an attacker could inject arbitrary headers. I know an attacker can control the ?redirect=
parameter via something like https://localhost:8080/oauth2/redirect?redirect=https://evil.com, but that's useless if they can't control the request headers too. Here's what would happen if an attacker were to try to pull this off:
- Attacker tricks user into visiting https://localhost:8080/oauth2/redirect?redirect=https://evil.com
- Argo stores
https://evil.com
in the cookie associated with thestate
parameter and redirects the user to the OAuth2 provider - User logs in
- OAuth2 provider redirects user to https://localhost:8080/oauth2/callback
- Argo reads the cookie and rejects
https://evil.com
because it doesn't match request headers, so it redirects them to https://localhost:8080/
Even if the attacker can execute Javascript and send AJAX requests, the CORS policy won't allow it. Try running this in a DevTools console at https://example.com :
fetch('https://localhost:8080/oauth2/redirect?redirect=https://evil.com', {headers: {"X-Forwarded-Host": "evil.com"}})
You should see an error like this:
Access to fetch at 'https://localhost:8080/oauth2/redirect?redirect=https://evil.com' from origin 'https://example.com' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: Redirect is not allowed for a preflight request.
Can you give a concrete example of the kind of attack you're thinking of?
Thanks Mason. I requested this over Slack as it is would make it easier to review and backport, and is a general best practice for smaller PRs. |
This adds a new `make start UI_SECURE=true` flag that can be used to test Argo with a TLS termination proxy in front, which is needed for testing argoproj#13609. Also, `make start SECURE=true UI=true` was broken because `ui/src/app/webpack.config.js` wasn't respecting the `ARGO_SECURE` flag, so I fixed that too. Note that `make start UI_SECURE=true PROFILE=sso` doesn't work without manually changing the `redirectUrl` here: https://github.com/argoproj/argo-workflows/blob/29ea3518a30b2b605290d0ba38f95bfb75a2901f/manifests/quick-start/sso/overlays/workflow-controller-configmap.yaml#L15 I tried to see if I could automate this, but I couldn't figure out a clean way of doing so. I guess I could create a new profile called `sso-secure` under `test/e2e/manifests/`? Signed-off-by: Mason Malone <[email protected]>
/retest |
/retest |
/retest |
3 similar comments
/retest |
/retest |
/retest |
Fixes #13031
Motivation
When running Argo in plain text mode behind a TLS termination proxy with SSO enabled, you will always be redirected to the workflows page after logging in, regardless of which page you came from. This is happening because
server/auth/sso/sso.go
is using the protocol and host of the request to determine the redirect URL, which can be different than the original request when a proxy is involved.Modifications
The fix involves reading from the X-Forwarded-Host and X-Forwarded-Proto headers when present. This is almost the same as the fix that
oauth2-proxy
made when they faced a similar issue in oauth2-proxy/oauth2-proxy#957. The only major difference is thatoauth2-proxy
only reads those headers when the--reverse-proxy
option is set, but I didn't add a corresponding option here. The only reason I can think of for doing so is to preserve backwards-compatibility, since it's theoretically possible this could break someone's setup. I can't think of any real-world situations where that can happen, but let me know if I should put this behind a option and I'm happy to do so.Verification
Testing instructions:
make start UI_SECURE=true PROFILE=sso
Screen.Recording.2024-09-16.at.9.56.43.AM.mov