[RFC] Rationalise OSM API authentication: OAuth 2 only, one authorization flow, one place to enter the parameters

[wangi]

This is a request for comments before any code is written. It grows out of #24889, #24890 and #24925, which are all the same underlying problem seen from three directions: authenticating against an openstreetmap-website instance that is not openstreetmap.org or openhistoricalmap.org.

Prerequisites. Two bug fixes should land first. They are independent of each other and can go in either order, but both are assumed by what follows.

• #24907 — a trailing slash on the API URL breaks OAuth authentication. It normalises the API URL in one place, which everything below relies on: there is no point agreeing on a single key for the OAuth parameters and the token while https://example.org/api and https://example.org/api/ are still two different keys.
• #24925 — authentication on upload fails against a server with no built-in client id. The OAuth parameters are stored under the API URL but were looked up with the host of the request, so the lookup missed and JOSM opened the browser with an empty client_id. Fixed there; the reason it belongs here is that nothing stopped the two keys drifting apart in the first place, and the next caller holding only one of them will hit the same wall. Settling that is point (d) below.

1. What the code looks like today

Everything belonging to a server is keyed two different ways. OAuth20Parameters.rememberPreferences() stores the parameters under the API URL, while OAuthAccessTokenHolder and CredentialsManager key the token by host, since getAccessToken()/setAccessToken() reduce whatever they are given with URI.create(api).getHost(). A caller therefore has to know which of the two it is holding, and nothing says which. #24925 is what that costs in practice.

Three authorization procedures are offered by AuthorizationProcedure, but:

• SEMI_AUTOMATIC is dead. It is referenced only by its own getText() and getDescription(). OAuthAuthorizationWizard.refreshAuthorisationProcedurePanel() throws UnsupportedOperationException for it, and nothing constructs a wizard with it.
• FULLY_AUTOMATIC does not use the wizard window at all. showDialog() short-circuits straight to authorize() for OAuth 2, so the wizard, its header panel, its "Accept Access Token" button and FullyAutomaticAuthorizationUI only ever appear for MANUALLY.
• MANUALLY is expert-only (ExpertToggleAction.addVisibilitySwitcher) and duplicates the advanced parameters panel inside a second tab of its own window.

OsmConnection.OAuthAccessTokenFetcher, the static fetcher field and OAuthAuthorizationWizard.obtainAccessToken(URL) are also dead: the fetcher is assigned in MainApplication and in two test harnesses but never invoked, because the upload path calls obtainOAuth20Token() directly.

AdvancedOAuthPropertiesPanel has problems of its own:

• oauth.settings.use-default is a single global boolean, not per API URL, so the choice made for one server is applied to the next one.
• When it is ticked, rememberPreferences() calls OAuthParameters.createDefault() with no argument, i.e. the parameters of the default OSM API, and stores them under the default OSM API's key, whichever server is actually being configured.
• There is no field for the access token; that lives only in the separate manual window.
• setChildComponentsEnabled() only walks JosmTextField and JLabel children.
• The field labelled "Redirect URL" is still called tfRequestTokenURL, left over from OAuth 1.0a.

Finally, the endpoint defaults are a guess. OAuthParameters.getDefaultOAuth20Parameters() first tries RFC 8414 discovery at <scheme>://<host>/.well-known/oauth-authorization-server, which openstreetmap-website does answer — for openstreetmap.org it returns

"authorization_endpoint":"https://www.openstreetmap.org/oauth2/authorize"
"token_endpoint":"https://www.openstreetmap.org/oauth2/token"

— but if that request fails, or the resource is marked offline, the fallback is OAuth20Parameters(clientId, secret, baseUrl, …) with baseUrl = the API URL, which produces <api url>/authorize and <api url>/token. Those are wrong for every openstreetmap-website instance, so whether a third-party server works at all currently depends on whether one HTTP request to a well-known URL succeeded, and the user is given no hint that this is what happened.

2. Proposal

(a) Drop Basic Authentication as an OSM API authentication method. Remove the radio buttons, BasicAuthenticationPreferencesPanel, addBasicAuthorizationHeader(), the "basic" branch of OsmConnection.addAuth(), the username/password branch of MessageNotifier, and UserIdentityManager.initFromPreferences()'s username path. The preference osm-server.auth-method becomes redundant.

CredentialsManager, CredentialsAgent, JosmPreferencesCredentialAgent and DefaultAuthenticator stay: they also serve proxy authentication and HTTP authentication for other hosts such as imagery servers. Only the OSM-API-as-basic-auth path goes.

Note this is already half-done: AuthenticationPreferencesPanel.updateAcceptableAuthenticationMethods() disables the basic option for the default OSM API, because openstreetmap.org dropped it.

On the servers we can speak for, nothing is left that needs it. openstreetmap.org has not accepted basic authentication for years; OpenGeofiction has moved to OAuth 2; and Arhet, the remaining third-party instance we know of, can speak OAuth 2 once the RFC 8414 .well-known file is in place. That last case is worth noting for point (e) below: publishing /.well-known/oauth-authorization-server is the supported way for a third-party instance to work with JOSM without needing a client id built into JOSM, and it is not documented anywhere a server operator would find it.

(a2) Remove what is left of OAuth 1.0a. The mechanism is already gone — OAuthVersion has only OAuth20 and OAuth21 — but the vocabulary and some classes outlived it:

• OsmPrivileges and OsmPrivilegesPanel: the 1.0a privilege checkboxes. OAuth 2 uses OsmScopes instead. OsmPrivilegesPanel is used only by FullyAutomaticAuthorizationUI, and OsmPrivileges only by that panel and its own test, so both go with (b)/(c).
• FullyAutomaticPropertiesPanel: the username/password panel of the old screen-scraping flow. Already completely unused — referenced only by its own unit test.
• AccessTokenInfoPanel: used only by FullyAutomaticAuthorizationUI.
• The preference keys oauth.access-token.key and oauth.access-token.secret are 1.0a token storage. They are still registered as sensitive keys in AbstractPreferences and still handled in UserIdentityManager.preferenceChanged(), but nothing writes them any more.
• Stale wording: OAuthAuthenticationPreferencesPanel's class javadoc still says "The preferences panel for the OAuth 1.0a preferences"; AuthenticationPreferencesPanel throws "One of OAuth 2.0, OAuth 1.0a, or Basic authentication must be checked"; AdvancedOAuthPropertiesPanel names its fields tfConsumerKey / tfConsumerSecret although they are labelled Client ID and Client Secret, and tfRequestTokenURL although it is labelled Redirect URL.
• OAuthAuthenticationPreferencesPanel.AlreadyAuthorisedPanel adds a JLabel reading "Access Token Secret:" with no field after it. OAuth 2 tokens have no secret; the label is a leftover and is visible in the preferences today.

(b) and (c) One authorization flow, no wizard window. Delete AuthorizationProcedure, OAuthAuthorizationWizard, FullyAutomaticAuthorizationUI, FullyAutomaticPropertiesPanel, ManualAuthorizationUI, AbstractAuthorizationUI, AccessTokenInfoPanel, OsmPrivilegesPanel and the dead OAuthAccessTokenFetcher. What remains is one "Authorize now" button that runs the browser-based OAuth 2 dance with the parameters currently shown in the panel.

(d) Access token becomes a field of the advanced parameters. Add "Access Token" to AdvancedOAuthPropertiesPanel, next to Client ID, Client Secret, Redirect URI, Authorize URL and Access Token URL. Show the whole group whenever "Use default settings" is unticked, rather than hiding it behind a second checkbox. Pasting a token then stores it against the same server as the parameters it was pasted next to, rather than depending on which of the two keys the caller happened to be holding.

While doing this, settle on one key for everything belonging to a server. The token is keyed by host today and the parameters by API URL; with the API URL normalised by #24907 either works, but it must be the same one in both places and in OsmConnection.

(e) Tooltips. Each field explains what it holds and where it comes from, e.g.:

• Client ID — the Client ID of an OAuth 2 application registered on this server. On an openstreetmap-website server, register it under My Settings > OAuth 2 applications.
• Client Secret — only needed if the application is registered as confidential. The applications JOSM has built in are public clients and have none.
• Redirect URI — must match exactly the redirect URI registered with the application. JOSM listens on http://127.0.0.1:8111/oauth_authorization, which needs remote control enabled.
• Authorize URL / Access Token URL — taken from the server's RFC 8414 metadata when it publishes any; on openstreetmap-website these are <site>/oauth2/authorize and <site>/oauth2/token. Note <site>, not the API URL.
• Access Token — paste a token issued outside JOSM instead of pressing "Authorize now".

The permission scopes JOSM requests should also be listed somewhere, since the application has to be registered with at least read_prefs, write_api, write_notes, read_gpx and write_gpx.

(f) Finish off authorization on upload. #24925 makes it work and stops it opening a browser when the client id is empty, but it leaves the user with the generic "no Access Token configured, open the Preferences Dialog" message. With the access token and the parameters in one place (d), this can become a message that says which server has no client id and offers to open the authentication preferences there and then.

Two smaller things in the same method: obtainOAuth20Token() saves the token under OsmApi.getOsmApi().getServerUrl() while the caller reads it back with the request URL — harmless today only because the holder reduces both to the host, and another reason to settle (d) — and the while (done.getCount() >= 0 && counter < 5) loop condition is always true, so it should just be a bounded await.

3. What we would like feedback on

1. Is dropping basic authentication acceptable? openstreetmap.org, openhistoricalmap.org and OpenGeofiction no longer need it, and the one other instance we know of can publish the RFC 8414 metadata and use OAuth 2 instead. We cannot survey private deployments, though, and osm-server.username / osm-server.password may be read by plugins. If it goes, what migration do you want for a profile that still has osm-server.auth-method=basic — switch it silently to OAuth 2, or warn once on startup?
2. Is deleting the wizard and the manual window acceptable? Two of the three procedures are already dead or expert-only, but these are public classes and a plugin may reference them. Is the usual deprecation cycle wanted here, or is removal fine given @since churn in this area?
3. Host or API URL as the key for tokens and parameters — which would you prefer?
4. Should oauth.settings.use-default become per API URL? Today one global flag decides it for every server.
5. What should happen when RFC 8414 discovery fails? This matters more once basic authentication is gone, because discovery becomes the main way a third-party instance works without a client id built into JOSM. Keep guessing <api url>/authorize, which is wrong for every openstreetmap-website instance? Guess <site>/oauth2/authorize instead? Or leave the fields empty and tell the user the endpoints must be entered by hand?
6. Would you accept a short wiki page for server operators saying what an openstreetmap-website instance has to do to work with JOSM: publish /.well-known/oauth-authorization-server, register a public client with redirect URI http://127.0.0.1:8111/oauth_authorization, and grant the scopes JOSM asks for? We can write it up as part of this work.

4. Out of scope

OAuth20Token.sign() compares hosts with this.oauthParameters.getApiUrl().contains(host), so a token for api.openstreetmap.org would also sign a request to the host openstreetmap.org. That is pre-existing and security-adjacent; it deserves its own ticket rather than being folded in here.