HTTP 500 error when trying to get access-token from master.apis.dev.openstreetmap.org with "openid" scope

[hauke96]

URL

https://master.apis.dev.openstreetmap.org/oauth2/token

How to reproduce the issue?

It seems that adding the scope openid to the list of requested scopes returns a HTTP 500 error on the https://master.apis.dev.openstreetmap.org/oauth2/token endpoint. Yes, I gave permission to that scope and yes I used all correct client-id and -secret values. Removing openid from the list of wanted scopes solves the problem.

The error does not appear on the productive www.openstreetmap.org server!

This bug does not affect me (and would have a very low priority for me personally), I just saw it by accident.

Reproduce

Requires python3 authlib dependency:

from authlib.integrations.requests_client import OAuth2Session

# dev server:
client_id = "..."
client_secret = "..."
domain = "master.apis.dev"

# prod server:
#client_id = "..."
#client_secret = "..."
#domain = "www"

redirect_uri = 'http://127.0.0.1:8000/callback'

# Remove the "openid" entry here and it'll work
scope = ["read_prefs", "openid"]

oauth = OAuth2Session(client_id=client_id, redirect_uri=redirect_uri, scope=scope)

authorization_url, state = oauth.create_authorization_url('https://'+domain+'.openstreetmap.org/oauth2/authorize')
print("Please visit:\n" + authorization_url)
print("")
authorization_response = input('Enter the redirect url from your browser and paste it here:\n')
token = oauth.fetch_token(
        'https://'+domain+'.openstreetmap.org/oauth2/token',
        authorization_response=authorization_response,
        client_secret=client_secret)
print("")
print("Access token is:")
print(token["access_token"])

1. Execute the script
2. Click on the presented link to open the OSM login form and grant access
3. The browser will redirect to http://127.0.0.1:8000/callback?..., copy the whole url from the browsers address bar, paste it into the terminal and press Enter
4. If everything works, the access token should appear. In the above script, the error message requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: https://master.apis.dev.openstreetmap.org/oauth2/token comes up.

This situation also appears using golang and the golang/oauth2 library.

Screenshot(s) or anything else?

No response

[mmd-osm]

OIDC needs a signing key in its config, which may simply be still missing on the dev instance.

Here’s the respective setting for production: https://github.com/openstreetmap/chef/blob/9c82d0eb2c75d1f5eb9b0689c5e3bed194fd1e0b/cookbooks/web/recipes/rails.rb#L151

I also hit this issue when testing in JOSM before, but forgot again to raise an issue, so thanks for doing that.

Also it’s really more of an operations issue, and would be better served there in the operations repo. No worries, this issue can be easily moved across.

[Zaczero]

I find it interesting how such an oversight passed the testing server.

[hauke96]

Actually, right now, I get a 404 on https://master.apis.dev.openstreetmap.org/oauth2/token

It's a POST request with url-encoded form body that looks like this when using cURL (with parameters in angled brackets):

curl --location 'https://master.apis.dev.openstreetmap.org/oauth2/token?grant_type=authorization_code&code=<the-code-from-the-oauth-callback>&redirect_uri=<the-redirect-url-to-your-app>' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'client_id=<the-client-id>' \
--data-urlencode 'client_secret=<the-client-secret>'

That request (I just tried it with meaningless dummy values) works and returns the expected error response (but with an expected 400 and not 404 status code):

{"error":"invalid_client","error_description":"La autenticación del cliente ha fallado por cliente desconocido, cliente no autenticado, o método de autenticación incompatible."}

Interestingly in Spanish ... well ... at least something 🤷

[mmd-osm]

@tomhughes : could you please take a look at the doorkeeper_signing_key setting for master.apis.dev.openstreetmap.org?

I wanted to start testing JOSM OAuth2 settings with @tsmock, which is not possible as long as https://master.apis.dev.openstreetmap.org/.well-known/oauth-authorization-server doesn't work.

[tomhughes]

Thanks for actually explaining what the problem is - it's not going to be an easy thing to fix though so I'll need to have a think about how to do it.

[mmd-osm]

My assumption here is that doorkeeper-openid_connect gem depends on the signing key to work properly. Once I configured a signing key in my local settings, .well-known/oauth-authorization-server was also available, which is a prerequisite for JOSM.

[tomhughes]

Sure, the issue is how do we automate the generation of those keys for dev instances and ensure they persist properly.

[tomhughes]

Hopefully openstreetmap/chef@1859ef7 has fixed this.