[PATCH][RFC] OSM OAuth 1.0a/Basic auth deprecation and removal

[taylor.smock]

pnorman has indicated that they are now looking at ​deprecating HTTP Basic Auth and OAuth 1.0a.

Do we want to start hiding the Basic Auth and OAuth 1.0a authentication methods from new users in a few months?

OSM will be shutting down OAuth 1.0a and HTTP Basic Auth starting June 01, 2024.
​https://community.openstreetmap.org/t/oauth-1-0a-and-http-basic-auth-shutdown/108490

[taylor.smock]

From Grant Slater (Firefishy) on OSMUS Slack:

Polite reminder for JOSM dev team: The OSM ops team are trying to deprecate HTTP Basic Auth and OAuth 1.0a. JOSM is in 1st place for current use of OAuth 1. Additional detail here: ​https://github.com/openstreetmap/operations/issues/867

A response in that issue from tomhughes indicates that it might be wise to view those authentication methods as deprecated for removal now, with removal coming when sufficient users have left them.

As of writing, pnorman said:

Over the last week 5% of API calls were HTTP Basic Auth, 23% OAuth 1.0a, and 73% OAuth 2.0

We are currently 27.8% of the API calls using OAuth 1.0a. We might want to start hiding Basic Auth/OAuth 1.0a from users now, if they are not currently using them.

[taylor.smock]

Drop support for Basic Auth and OAuth 1.0a

[taylor.smock]

In 18828/josm:

See #22810: OSM OAuth 1.0a/Basic auth deprecation and removal

This prevents users from using Basic Authentication or OAuth 1.0a unless one of the following is true:

• They were previously using the authentication method
• They have enabled Expert Mode
• They are not using the default OSM API

[taylor.smock]

Notes on the patch:

• We need to add a new osm.oauth2 run argument for the CI
  • We cannot drop osm.username -- it is used by some tests. osm.password can probably be dropped.
• It drops support for Basic Authentication and OAuth 1.0a. This may affect very old third-party installations of the OSM server stack (but they should probably update; OAuth 2 has been supported by the OSM website since June 2021 (>2 years))
• It includes changes for some plugins in our svn
• It removes signpost (for OAuth 1.0) from our dependencies
• It hardcodes a client id for the OSM dev api server -- we can either leave it as is, or the someone with josm_test/josmeditor dev api user credentials can give me a different client id.
  • Do we want to reach out to major OSM forks (like OpenHistoricalMap) and give them the opportunity to have client ids hardcoded for them?
• OAuth 2 now supports custom variables (I could probably make this work while keeping OAuth 1.0a, but I don't think it would be worth the effort to do that and keep OAuth 1.0a)

I'll make an Upcoming Changes section on the startup page so that people know we are dropping basic auth and oauth 1.0a.

Probably something like

• Upcoming Changes
  • Basic Authentication and OAuth 1.0a support will be removed from JOSM in January 2024. See #22810 for details.

[stoecker]

It drops support for Basic Authentication

I'd leave that in, but remove all the config stuff and only leave variables as expert settings.

Rationale: Now with HTTPS Basic authentication isn't really a bad method anymore and for servers beside the main server will always be a proper choice.

[taylor.smock]

I'd leave that in, but remove all the config stuff and only leave variables as expert settings.

Works for me. Caveat: they aren't visible in expert settings.

Rationale: Now with HTTPS Basic authentication isn't really a bad method anymore and for servers beside the main server will always be a proper choice.

I could probably just leave the Basic authentication option in the UI, but only enable it if the server isn't the main OSM API.
With that said, I think most OSM-like servers/databases just use the OSM webstack, so they could use OAuth 2.

As a specific example, OHM supports OAuth 2, so all they (should) need to do is have a clientId available in their setup instructions.

[LuigiCotocea]

Removing basing authentication is a bad idea, sites like OpenGeoFiction need basic authentication because of not working, prehaps i should contact the site admin to tell them to implement the new Auth 2.0 to their site's api!

[taylor.smock]

Replying to LuigiCotocea:

Removing basing authentication is a bad idea, sites like OpenGeoFiction need basic authentication because of not working, prehaps i should contact the site admin to tell them to implement the new Auth 2.0 to their site's api!

You might want to try replacing opengeofiction.net/api with opengeofiction.net/oauth2. I have not yet patched in good "defaults" for non-OpenStreetMap website OSM-like authentication APIs.

EDIT: Also note that you will need an OAuth2 clientId, which I have not yet made available in the UI. If you want to test out a version of JOSM with the patch applied, the OpenHistoricalMap guys uploaded a copy of JOSM with the patch applied from OSMUS slack to ​https://github.com/OpenHistoricalMap/issues/issues/610 .

[anonymous]

I suggest you add instructions to the JOSM banner on how "existing users switch to it". I had to dig to find where I do this... save people some work, else few will do it until you just FAIL their login.

OpenStreetMap is deprecating Basic Authentication and OAuth 1.0a. JOSM will be removing OAuth 1.0a support in early 2024, and may hide or remove Basic Authentication support at the same time. We recommend that all new users use OAuth 2.0 and existing users switch to it. See #22810 and the OpenStreetMap Operations ticket #867 for further details.

Instead:
OpenStreetMap is deprecating Basic Authentication and OAuth 1.0a. JOSM will be removing OAuth 1.0a support in early 2024, and may hide or remove Basic Authentication support at the same time. We recommend that all new users use OAuth 2.0 and existing users switch to it (Edit>Preferences>OSM Server>Use OAuth 2.0>New Access Token). See #22810 and the OpenStreetMap Operations ticket #867 for further details.

[taylor.smock]

@anonymous: Thank you for the feedback. I'll update wiki:Help/Preferences/Connection#Authentication and link to that.

[luciano@…]

Replying to taylor.smock:

Replying to LuigiCotocea:

Removing basing authentication is a bad idea, sites like OpenGeoFiction need basic authentication because of not working, prehaps i should contact the site admin to tell them to implement the new Auth 2.0 to their site's api!

You might want to try replacing opengeofiction.net/api with opengeofiction.net/oauth2. I have not yet patched in good "defaults" for non-OpenStreetMap website OSM-like authentication APIs.

EDIT: Also note that you will need an OAuth2 clientId, which I have not yet made available in the UI. If you want to test out a version of JOSM with the patch applied, the OpenHistoricalMap guys uploaded a copy of JOSM with the patch applied from OSMUS slack to ​https://github.com/OpenHistoricalMap/issues/issues/610 .

Speaking in my role as current site-host/tech-admin for OpenGeofiction: we are using an OLD fork of the OSM stack (pre-Oauth2 enabled, about 3 years old). Upgrading is non-trivial because of adaptations that were made to the code base to make OGF possible (non-earth setting, changed osm-website, even some small changes to database) - we are NOT professional coders but just hobbyists! Updating to current OSM stack would require re-factoring all these aspects of our site.

I'd urge whatever is done with JOSM to retain ability to use basic authentication for non-OSM api's.

Alternatively we will have to create a fork of JOSM ("frozen in time") to support our community.

Thanks!

[taylor.smock]

we are using an OLD fork of the OSM stack (pre-Oauth2 enabled, about 3 years old).

Are you certain about the OAuth2 statement? It looks like you have a working ​/oauth2/authorize endpoint.

For reference, ​opengeofiction-website (hopefully I have the "right" repo) has code for supporting ​oauth2.

Anyway, I would strongly recommend upgrading for security reasons. I don't think you want to wake up one morning and not have a usable database. If nothing else, do a monthly rebase (so rebase onto whatever commit the openstreetmap website had onto 2021-10-01, then after a week, rebase onto 2021-11-01, and so on, until you start getting non-trivial conflicts). The major problem seems to be the localization stuff; your repo moved them in f90677fae2c6bc3bc55425e1e1f9ab23525e6a7d and 6bb5fd951ec5af53c95908e1abeb066829b89a44. If you revert the moving of those translations, you'll have a much easier time rebasing.

[anonymous]

Replying to taylor.smock:

we are using an OLD fork of the OSM stack (pre-Oauth2 enabled, about 3 years old).

Are you certain about the OAuth2 statement? It looks like you have a working ​/oauth2/authorize endpoint.

For reference, ​opengeofiction-website (hopefully I have the "right" repo) has code for supporting ​oauth2.

Anyway, I would strongly recommend upgrading for security reasons. I don't think you want to wake up one morning and not have a usable database. If nothing else, do a monthly rebase (so rebase onto whatever commit the openstreetmap website had onto 2021-10-01, then after a week, rebase onto 2021-11-01, and so on, until you start getting non-trivial conflicts). The major problem seems to be the localization stuff; your repo moved them in f90677fae2c6bc3bc55425e1e1f9ab23525e6a7d and 6bb5fd951ec5af53c95908e1abeb066829b89a44. If you revert the moving of those translations, you'll have a much easier time rebasing.

Wow, I appreciate the fast response! And appreciate your points.

I had reports from users that Oauth2 wasn't working, and when I attempted to test it, I got Ruby on Rails error pages from our website. So I just assumed I was running a pre-Oauth2 version. But it could be that somehow things just aren't configured properly.

This OGF link is the one opened by JOSM's "Authorize Now (Semi-Automatic)" button: ​https://opengeofiction.net/api/authorize

As you can see, it leads to error - at least from my two desktops, running JOSM 18822, with and without context of an already logged in OGF account.

If our fork supposedly supports Oauth2, then I suspect issues with our modified routes.rb file.

I need to emphasize, again, we are JUST hobbyists! My own very rusty IT background is in SQL and data warehouse design, not Ruby-on-Rails nor Java nor any front-end website work. So in fact most of what you describe about rebasing our fork is over my head - I just don't have much experience or knowledge or success with using git in a professional context. I understand the principles involved, I just don't have any knowledge about how it all works at the level of implementation. I'm not sure I have the time to learn - I mean, I'm interested, but as a hobby, work on maintaining or improving OGF has to come after my day-to-day work and life obligations.

I do agree that the translations are a major sticking point for us. We have disabled internationalization support because OSM uses a pay-to-play internationalization tool (I can't remember the name) which we can't afford, and I have been unable to find any open source and easy to use internationalization maintenance tool for Ruby on Rails. Further, because the terms "OSM" and "Openstreetmap" are "hard-coded" in the osm-website application and the translation pages, there were extensive and painful "find-and-replace" operations across the code-base (which remain imperfect).

Anyway, I appreciate your having taken the time to respond, and all the work the JOSM team does to make an excellent tool.

[taylor.smock]

This OGF link is the one opened by JOSM's "Authorize Now (Semi-Automatic)" button: ​​https://opengeofiction.net/api/authorize

One of the known problems with the code in JOSM as it currently exists is that there isn't a way to set a custom endpoint in the UI. The patch adds that ability. The /authorize endpoint is just automatically added on to all non-OSM endpoints if nothing specific was provided.

when I attempted to test it

I would strongly recommend testing with jar that the OHM folks uploaded at ​https://github.com/OpenHistoricalMap/issues/issues/610 .

I'll see if I can get the custom endpoint settings working with current code and commit that sometime next week. If I forget, poke me!

[luciano@…]

Replying to taylor.smock:

This OGF link is the one opened by JOSM's "Authorize Now (Semi-Automatic)" button: ​​https://opengeofiction.net/api/authorize

One of the known problems with the code in JOSM as it currently exists is that there isn't a way to set a custom endpoint in the UI. The patch adds that ability. The /authorize endpoint is just automatically added on to all non-OSM endpoints if nothing specific was provided.

when I attempted to test it

I would strongly recommend testing with jar that the OHM folks uploaded at ​https://github.com/OpenHistoricalMap/issues/issues/610 .

I'll see if I can get the custom endpoint settings working with current code and commit that sometime next week. If I forget, poke me!

Wow, actually, your comment clarified for me, a little bit, what was going on.

As a test, I put another line in the routes.rb file, redirecting /api/authorize => /oauth2/authorize

That gets us past the error, when clicking the "Authorize Now" button, and the parameters are carried over.

We're still missing the client_id, however. That has to come from JOSM, I guess? Or is it something that gets "registered" in the OGF code somewhere?

Thanks again for rapid responses. I'll be offline now rest of the day.

[taylor.smock]

We're still missing the client_id, however. That has to come from JOSM, I guess? Or is it something that gets "registered" in the OGF code somewhere?

Technically speaking, it should come from the OGF website. There should be an OAuth 2 applications tab in your account settings (​https://opengeofiction.net/oauth2/applications should take you there).

As a test, I put another line in the routes.rb file, redirecting /api/authorize => /oauth2/authorize

You shouldn't have to do this in the future. A "better" thing to do would be to add an endpoint returning an ​RFC8414 compliant json. See ​https://www.openstreetmap.org/.well-known/oauth-authorization-server for an example. IIRC, we (JOSM) only need authorization_endpoint and token_endpoint.

I'll be offline now rest of the day.

Have a good evening.

[alt.people.davidcalman@…]

Don't do it until OSM's backup servers and OHM are compatible. This requires fixing #23297.

[skyper]

Ticket #23297 has been marked as a duplicate of this ticket.

[taylor.smock]

Ticket retargeted after milestone closed

[taylor.smock]

Ticket #23475 has been marked as a duplicate of this ticket.

[taylor.smock]

Update patch

[taylor.smock]

I intend to apply attachment:22810.patch​ today or tomorrow; there is no point in keeping the OAuth 1.0a implementation around any longer (see #23475 -- something changed on the OSM side which made obtaining a new OAuth 1.0a token impossible).

[taylor.smock]

In 18991/josm:

Fix #22810: OSM OAuth 1.0a/Basic auth deprecation and removal

As of 2024-02-15, something changed in the OSM server configuration. This broke
our OAuth 1.0a implementation (see #23475). As such, we are removing OAuth 1.0a
from JOSM now instead of when the OSM server removes support in June 2024.

For third-party OpenStreetMap servers, the Basic Authentication method has been
kept. However, they should be made aware that it may be removed if a non-trivial
bug occurs with it. We highly recommend that the third-party servers update to
the current OpenStreetMap website implementation (if only for their own security).

Failing that, the third-party server can implement RFC8414. As of this commit,
we currently use the authorization_endpoint and token_endpoint fields.
To check and see if their third-party server implements RFC8414, they can go
to <server host>/.well-known/oauth-authorization-server.

Prominent third-party OpenStreetMap servers may give us a client id for their
specific server. That client id may be added to the hard-coded client id list
at maintainer discretion. At a minimum, the server must be publicly
available and have a significant user base.

[taylor.smock]

In 36211/osm:

See #22810: OSM OAuth 1.0a/Basic auth deprecation and removal

[taylor.smock]

In 36212/osm:

See #22810: OSM OAuth 1.0a/Basic auth deprecation and removal (dist)