id summary reporter owner description type status priority milestone component version resolution keywords cc 24928 [RFC] Rationalise OSM API authentication: one authorization flow, one place to enter the parameters wangi team "''Updated after the feedback in comment:2.'' This is a request for comments before any code is written. It grows out of #24889, #24890 and #24925, which are all the same underlying problem seen from three directions: authenticating against an openstreetmap-website instance that is not openstreetmap.org or openhistoricalmap.org. '''Prerequisites.''' Three bug fixes should land first. They are independent of each other and can go in any order, but all are assumed by what follows. * '''#24907''' — a trailing slash on the API URL breaks OAuth authentication. It normalises the API URL in one place, which everything below relies on: there is no point agreeing on a single key for the OAuth parameters and the token while `https://example.org/api` and `https://example.org/api/` are still two different keys. * '''#24925''' — authentication on upload fails against a server with no built-in client id. The OAuth parameters are stored under the API URL but were looked up with the host of the request, so the lookup missed and JOSM opened the browser with an empty `client_id`. Fixed there; the reason it belongs here is that nothing stopped the two keys drifting apart in the first place, and the next caller holding only one of them will hit the same wall. Settling that is point (d) below. * '''#24932''' — ""Remove token"" leaves the OAuth access token in the preferences. It fixes the method that (d) reworks, and what removing a token deletes has to be right before the token and the parameters share one key. == 1. What the code looks like today == '''Everything belonging to a server is keyed two different ways.''' `OAuth20Parameters.rememberPreferences()` stores the parameters under the API URL, while `OAuthAccessTokenHolder` and `CredentialsManager` key the token by host, since `getAccessToken()`/`setAccessToken()` reduce whatever they are given with `URI.create(api).getHost()`. A caller therefore has to know which of the two it is holding, and nothing says which. #24925 is what that costs in practice. Three authorization procedures are offered by `AuthorizationProcedure`, but: * '''`SEMI_AUTOMATIC` is dead.''' It is referenced only by its own `getText()` and `getDescription()`. `OAuthAuthorizationWizard.refreshAuthorisationProcedurePanel()` throws `UnsupportedOperationException` for it, and nothing constructs a wizard with it. * '''`FULLY_AUTOMATIC` does not use the wizard window at all.''' `showDialog()` short-circuits straight to `authorize()` for OAuth 2, so the wizard, its header panel, its ""Accept Access Token"" button and `FullyAutomaticAuthorizationUI` only ever appear for `MANUALLY`. * '''`MANUALLY` is expert-only''' (`ExpertToggleAction.addVisibilitySwitcher`) and duplicates the advanced parameters panel inside a second tab of its own window. `OsmConnection.OAuthAccessTokenFetcher`, the static `fetcher` field and `OAuthAuthorizationWizard.obtainAccessToken(URL)` are also dead: the fetcher is assigned in `MainApplication` and in two test harnesses but never invoked, because the upload path calls `obtainOAuth20Token()` directly. `AdvancedOAuthPropertiesPanel` has problems of its own: * `oauth.settings.use-default` is a '''single global boolean''', not per API URL, so the choice made for one server is applied to the next one. * When it is ticked, `rememberPreferences()` calls `OAuthParameters.createDefault()` '''with no argument''', i.e. the parameters of the default OSM API, and stores them under the default OSM API's key, whichever server is actually being configured. * There is no field for the access token; that lives only in the separate manual window. * `setChildComponentsEnabled()` only walks `JosmTextField` and `JLabel` children. * The field labelled ""Redirect URL"" is still called `tfRequestTokenURL`, left over from OAuth 1.0a. Finally, the endpoint defaults are a guess. `OAuthParameters.getDefaultOAuth20Parameters()` first tries RFC 8414 discovery at `:///.well-known/oauth-authorization-server`, which openstreetmap-website does answer — for openstreetmap.org it returns {{{ ""authorization_endpoint"":""https://www.openstreetmap.org/oauth2/authorize"" ""token_endpoint"":""https://www.openstreetmap.org/oauth2/token"" }}} — but if that request fails, or the resource is marked offline, the fallback is `OAuth20Parameters(clientId, secret, baseUrl, …)` with `baseUrl` = the API URL, which produces `/authorize` and `/token`. Those are wrong for '''every''' openstreetmap-website instance, so whether a third-party server works at all currently depends on whether one HTTP request to a well-known URL succeeded, and the user is given no hint that this is what happened. == 2. Proposal == '''(a) Basic Authentication for experts only.''' Keep it as an OSM API authentication method, as a quick way to test one's own server, but show the option only in expert mode — or when it is the method currently selected, so that a profile still using it does not lose sight of its own setting. As today, `AuthenticationPreferencesPanel.updateAcceptableAuthenticationMethods()` keeps it disabled for the default OSM API, which does not accept it. (Originally proposed for removal; changed after comment:2.) Publishing `/.well-known/oauth-authorization-server` is the supported way for a third-party instance to work with JOSM without needing a client id built into JOSM, and it is not documented anywhere a server operator would find it; see the wiki page in section 3. '''(a2) Remove what is left of OAuth 1.0a.''' The mechanism is already gone — `OAuthVersion` has only `OAuth20` and `OAuth21` — but the vocabulary and some classes outlived it: * `OsmPrivileges` and `OsmPrivilegesPanel`: the 1.0a privilege checkboxes. OAuth 2 uses `OsmScopes` instead. `OsmPrivilegesPanel` is used only by `FullyAutomaticAuthorizationUI`, and `OsmPrivileges` only by that panel and its own test, so both go with (b)/(c). * `FullyAutomaticPropertiesPanel`: the username/password panel of the old screen-scraping flow. '''Already completely unused''' — referenced only by its own unit test. * `AccessTokenInfoPanel`: used only by `FullyAutomaticAuthorizationUI`. * The preference keys `oauth.access-token.key` and `oauth.access-token.secret` are 1.0a token storage. They are still registered as sensitive keys in `AbstractPreferences` and still handled in `UserIdentityManager.preferenceChanged()`, but nothing writes them any more. * Stale wording: `OAuthAuthenticationPreferencesPanel`'s class javadoc still says ""The preferences panel for the OAuth 1.0a preferences""; `AuthenticationPreferencesPanel` throws ""One of OAuth 2.0, OAuth 1.0a, or Basic authentication must be checked""; `AdvancedOAuthPropertiesPanel` names its fields `tfConsumerKey` / `tfConsumerSecret` although they are labelled Client ID and Client Secret, and `tfRequestTokenURL` although it is labelled Redirect URL. * `OAuthAuthenticationPreferencesPanel.AlreadyAuthorisedPanel` adds a `JLabel` reading '''""Access Token Secret:"" with no field after it'''. OAuth 2 tokens have no secret; the label is a leftover and is visible in the preferences today. '''(b) and (c) One authorization flow, no wizard window.''' Delete `AuthorizationProcedure`, `OAuthAuthorizationWizard`, `FullyAutomaticAuthorizationUI`, `FullyAutomaticPropertiesPanel`, `ManualAuthorizationUI`, `AbstractAuthorizationUI`, `AccessTokenInfoPanel`, `OsmPrivilegesPanel` and the dead `OAuthAccessTokenFetcher`. What remains is one ""Authorize now"" button that runs the browser-based OAuth 2 dance with the parameters currently shown in the panel. The manual path stays: only its window goes, and a token obtained outside JOSM is pasted into the access token field of (d). '''(d) Access token becomes a field of the advanced parameters.''' Add ""Access Token"" to `AdvancedOAuthPropertiesPanel`, next to Client ID, Client Secret, Redirect URI, Authorize URL and Access Token URL. Show the whole group whenever ""Use default settings"" is unticked, rather than hiding it behind a second checkbox. Pasting a token then stores it against the same server as the parameters it was pasted next to, rather than depending on which of the two keys the caller happened to be holding. As the fallback for when everything else breaks, a pasted token must work with nothing else: no client id, no endpoints, no RFC 8414 discovery, no remote control and no browser. Today it cannot, because `OAuth20Token` requires complete `OAuth20Parameters`. While doing this, '''settle on one key''' for everything belonging to a server: the '''host'''. `JosmPreferencesCredentialAgent` already stores the token, and the parameters it was issued with, under `oauth.access-token.parameters.OAuth20.` — the same prefix `rememberPreferences()` writes to under the API URL, so a configured server ends up with two parameter entries. With the host, `rememberPreferences()` writes the entry the token store already uses, every stored token stays valid, and only custom parameters need reading once from their old key. Removing a token must then remove the token only, not the client id the user entered. The global `oauth.settings.use-default` goes too: ""use default settings"" simply means that no custom parameters are stored for that host. '''(e) Tooltips.''' Each field explains what it holds and where it comes from, e.g.: * '''Client ID''' — the Client ID of an OAuth 2 application registered on this server. On an openstreetmap-website server, register it under My Settings > OAuth 2 applications. * '''Client Secret''' — only needed if the application is registered as confidential. The applications JOSM has built in are public clients and have none. * '''Redirect URI''' — must match exactly the redirect URI registered with the application. JOSM listens on `http://127.0.0.1:8111/oauth_authorization`, which needs remote control enabled. * '''Authorize URL / Access Token URL''' — taken from the server's RFC 8414 metadata when it publishes any; on openstreetmap-website these are `/oauth2/authorize` and `/oauth2/token`. Note ``, not the API URL. When discovery fails they are pre-filled with exactly those, rather than today's `/authorize`, and stay editable. * '''Access Token''' — paste a token issued outside JOSM instead of pressing ""Authorize now"". The permission scopes JOSM requests should also be listed somewhere, since the application has to be registered with at least `read_prefs`, `write_api`, `write_notes`, `read_gpx` and `write_gpx`. '''(f) Finish off authorization on upload.''' #24925 makes it work and stops it opening a browser when the client id is empty, but it leaves the user with the generic ""no Access Token configured, open the Preferences Dialog"" message. With the access token and the parameters in one place (d), this can become a message that says which server has no client id and offers to open the authentication preferences there and then. Two smaller things in the same method: `obtainOAuth20Token()` saves the token under `OsmApi.getOsmApi().getServerUrl()` while the caller reads it back with the request URL — harmless today only because the holder reduces both to the host, and another reason to settle (d) — and the `while (done.getCount() >= 0 && counter < 5)` loop condition is always true, so it should just be a bounded `await`. == 3. Feedback == 1. '''Is dropping basic authentication acceptable?''' openstreetmap.org, openhistoricalmap.org and OpenGeofiction no longer need it, and the one other instance we know of can publish the RFC 8414 metadata and use OAuth 2 instead. We cannot survey private deployments, though, and `osm-server.username` / `osm-server.password` may be read by plugins. If it goes, what migration do you want for a profile that still has `osm-server.auth-method=basic` — switch it silently to OAuth 2, or warn once on startup? — '''No''': it stays, for experts; see (a). 2. '''Is deleting the wizard and the manual window acceptable?''' Two of the three procedures are already dead or expert-only, but these are public classes and a plugin may reference them. Is the usual deprecation cycle wanted here, or is removal fine given `@since` churn in this area? — '''A manual path must stay''': it does, as the access token field; see (b)/(c) and (d). Deprecation cycle or removal: '''still open'''. 3. '''Host or API URL as the key''' for tokens and parameters — which would you prefer? — '''Host'''; see (d). 4. Should `oauth.settings.use-default` become '''per API URL'''? Today one global flag decides it for every server. — '''Yes''', it falls out of (d). 5. '''What should happen when RFC 8414 discovery fails?''' This matters most for third-party servers, because discovery is the main way a third-party instance works without a client id built into JOSM. Keep guessing `/authorize`, which is wrong for every openstreetmap-website instance? Guess `/oauth2/authorize` instead? Or leave the fields empty and tell the user the endpoints must be entered by hand? — '''`/oauth2/…`, editable''', which is the manual fallback of question 2; see (e). 6. Would you accept a short '''wiki page for server operators''' saying what an openstreetmap-website instance has to do to work with JOSM: publish `/.well-known/oauth-authorization-server`, register a public client with redirect URI `http://127.0.0.1:8111/oauth_authorization`, and grant the scopes JOSM asks for? We can write it up as part of this work. — '''Yes'''. == 4. Out of scope == `OAuth20Token.sign()` compares hosts with `this.oauthParameters.getApiUrl().contains(host)`, so a token for `api.openstreetmap.org` would also sign a request to the host `openstreetmap.org`. That is pre-existing and security-adjacent; it deserves its own ticket rather than being folded in here." enhancement new normal Core oauth oauth2 authentication basic auth upload client_id third-party api