Modify

Opened 4 years ago

Last modified 3 months ago

#10033 new defect

Josm do not start in remote control from osm.org in https

Reported by: anonymous Owned by: team
Priority: normal Milestone: 18.05
Component: Core remotecontrol Version:
Keywords: https certificate localhost Cc: Lesath, simon04, naoliv, Stereo

Description

What steps will reproduce the problem?

  1. use firefox or chromium (Ubuntu 14.04)
  2. visit osm main page
  3. try to edit a area from menu...

What is the expected result?
Open edi area as usual

What happens instead?
No error and josm do not open and download the edit area... (josm is opened without any data loaded)

Please provide any additional information below. Attach a screenshot if
possible.

Repository Root: http://josm.openstreetmap.de/svn
Build-Date: 2014-05-16 01:37:38
Last Changed Author: Don-vip
Revision: 7134
Repository UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
URL: http://josm.openstreetmap.de/svn/trunk
Last Changed Date: 2014-05-15 20:03:47 +0200 (Thu, 15 May 2014)
Last Changed Rev: 7134

Identification: JOSM/1.5 (7134 it) Linux Ubuntu 14.04 LTS
Memory Usage: 169 MB / 455 MB (83 MB allocated, but free)
Java version: 1.7.0_55, Oracle Corporation, Java HotSpot(TM) Server VM
VM arguments: [-Xmx512M]

Plugin: AddrInterpolation (30416)
Plugin: CommandLine (30436)
Plugin: FastDraw (30416)
Plugin: FixAddresses (30416)
Plugin: ImportImagePlugin (30416)
Plugin: OpeningHoursEditor (30416)
Plugin: PicLayer (30436)
Plugin: SimplifyArea (30416)
Plugin: buildings_tools (30416)
Plugin: conflation (0.1.6)
Plugin: continuosDownload (28565)
Plugin: dataimport (30416)
Plugin: download_along (30416)
Plugin: geotools (30416)
Plugin: jts (30416)
Plugin: junctionchecking (30416)
Plugin: log4j (30416)
Plugin: mapdust (30416)
Plugin: merge-overlap (30416)
Plugin: mirrored_download (30436)
Plugin: namemanager (30416)
Plugin: notes (v0.9.2)
Plugin: opendata (30436)
Plugin: pbf (30416)
Plugin: poly (30416)
Plugin: print (30416)
Plugin: proj4j (30417)
Plugin: reltoolbox (30416)
Plugin: reverter (30436)
Plugin: scripting (30604)
Plugin: tageditor (30416)
Plugin: tagging-preset-tester (30416)
Plugin: terracer (30416)
Plugin: turnrestrictions (30416)
Plugin: undelete (30416)
Plugin: utilsplugin2 (30419)
Plugin: walkingpapers (30416)
Plugin: waydownloader (30416)
Plugin: waypoint_search (30416)
Plugin: wikipedia (30449)

Attachments (1)

remove_https_remote_control.diff (57.7 KB) - added by Don-vip 4 months ago.
what it looks like if we remove HTTP support for remote control

Download all attachments as: .zip

Change History (53)

comment:1 Changed 4 years ago by Don-vip

Component: CoreCore remotecontrol
Keywords: https certificate added; template_report removed

Indeed it seems at least chrome doesn't like our autosigned certificate. But we can't get a real one for localhost :D we need to find a way to make browser accept it.

comment:2 Changed 4 years ago by Don-vip

Good news, we can access system keystore from Java and add our own certificate: http://stackoverflow.com/a/5510555/2257172

At least for Windows, I don't know yet for Linux. Even less for Mac :)

comment:3 Changed 4 years ago by Don-vip

Keywords: localhost added
Milestone: 14.06
Summary: Josm do not start in remote control from browser linkJosm do not start in remote control from osm.org in https

comment:4 Changed 4 years ago by Don-vip

In 7206/josm:

see #10033 - allow remote control to work from osm.org in https on Windows systems by adding updated JOSM localhost certificate to Windows Root Certificates keystore

comment:5 Changed 4 years ago by bastiK

URL for testing: https://127.0.0.1:8112/load_and_zoom?left=8.19&right=8.20&top=48.605&bottom=48.590&select=node413602999

General info: http://www.chromium.org/Home/chromium-security/root-ca-policy

There is a KeyStore called PKCS11-NSS, but just copying your code doesn't work...

Last edited 4 years ago by Don-vip (previous) (diff)

comment:6 in reply to:  5 Changed 4 years ago by Don-vip

Replying to bastiK:

There is a KeyStore called PKCS11-NSS, but just copying your code doesn't work...

Yes it requires some configuration described here.

I have not yet read everything but as I understand, we need to build a configuration file like

name=<anything>
nssLibraryDirectory = <path to libnss>
nssSecmodDirectory = <path to nssdb> (/etc/pki/nssdb)
nssDbMode = readWrite

comment:7 Changed 4 years ago by Don-vip

Cc: Lesath added

@Lesath: Can you please tell me if the bug also affects Safari on Mac OSX? If yes, does this piece of code fix the problem if we add it in tools/PlatformOSX?

    @Override
    public void setupHttpsCertificate(KeyStore.PrivateKeyEntry privateKeyEntry)
            throws KeyStoreException, NoSuchAlgorithmException, CertificateException, IOException {
        KeyStore ks = KeyStore.getInstance("KeychainStore", "Apple");
        ks.load(null, null);
        Enumeration<String> en = ks.aliases();
        while (en.hasMoreElements()) {
            String alias = en.nextElement();
            Certificate c = ks.getCertificate(alias);
            if (ks.isKeyEntry(alias) && c.equals(privateKeyEntry.getCertificate())) {
                // JOSM certificate found, return
                return;
            }
        }
        // JOSM certificate not found, install it
        Main.info("Adding JOSM localhost certificate to Apple KeychainStore");
        ks.setEntry("josm_localhost", privateKeyEntry, new KeyStore.PasswordProtection("josm_ssl".toCharArray()));
    }
Last edited 4 years ago by Don-vip (previous) (diff)

comment:8 Changed 4 years ago by Lesath

Hi Don-vip,

in Safari it works Out-Of-The-Box somewhat - Safari warns the user about a not trusted certificate for "127.0.0.1" with/without the patch. If you click on "Continue" in Safari it will open the connection.

With Firefox in won't work - it says that the Remote Control Mode is not activated with/without the patch. I'm not sure that I've imported the right classes - so here is the code that compiles in my Mac:

    @Override
    public void setupHttpsCertificate(KeyStore.PrivateKeyEntry privateKeyEntry)
            throws KeyStoreException, NoSuchAlgorithmException, CertificateException, IOException {
        KeyStore ks = null;
        try {
            ks = KeyStore.getInstance("KeychainStore", "Apple");
        } catch (NoSuchProviderException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
        ks.load(null, null);
        Enumeration<String> en = ks.aliases();
        while (en.hasMoreElements()) {
            String alias = en.nextElement();
            Certificate c = ks.getCertificate(alias);
            if (ks.isKeyEntry(alias) && c.equals(privateKeyEntry.getCertificate())) {
                // JOSM certificate found, return
                return;
            }
        }
        // JOSM certificate not found, install it
        Main.info("Adding JOSM localhost certificate to Apple KeychainStore");
        ks.setEntry("josm_localhost", privateKeyEntry, new KeyStore.PasswordProtection("josm_ssl".toCharArray()));
    }

I've debugged the code, and it adds the certificate, but I don't know why it doesn't work - currently I've got no time to debug it.

comment:9 Changed 4 years ago by Don-vip

Milestone: 14.0614.07

Move tickets where work remains to next milestone

comment:10 Changed 4 years ago by Don-vip

In 7335/josm:

see #10230, see #10033 - big rework of HTTPS support for Remote Control:

  • HTTPS disabled by default, must be enabled in remote control preferences
  • Old certificate and private key removed from jar and Windows keystore if found, even if remote control disabled
  • New certificate generated at runtime with critical X509 extensions BasicConstraints (non-CA certificate), ExtendedKeyUsage (usage restriction for TLS server sessions)
  • New passwords generated at runtime (but stored in clear in user preferences)
  • Private key is no longer stored in Windows keystore (only certificate)

comment:11 Changed 4 years ago by Don-vip

In 7336/josm:

see #10230, see #10033 - fix unit test

comment:12 Changed 4 years ago by Don-vip

In 7337/josm:

see #10230, see #10033 - SAN tweaks + fix unit test (for real?)

comment:13 Changed 4 years ago by Don-vip

In 7338/josm:

see #10230, see #10033 - JDK8 compatibility

comment:14 Changed 4 years ago by Don-vip

Milestone: 14.0714.08

I won't be able to finish the entire subject on this release.

comment:15 Changed 4 years ago by Don-vip

In 7342/josm:

see #10230, see #10033 - fix certificate detection

comment:16 Changed 4 years ago by Don-vip

In 7343/josm:

see #10230, see #10033 - add "Install/uninstall certificate" buttons in remote control preferences (Windows only)

comment:17 in reply to:  14 Changed 4 years ago by bastiK

Replying to Don-vip:

I won't be able to finish the entire subject on this release.

I think it's no problem to delay it for a week or so, if that helps ...

comment:18 Changed 4 years ago by Don-vip

Cc: simon04 added

I will need far more than a week :) Concerning Firefox (all platforms) and Linux for example, I fear we need to create a plugin :( Plus, I'm taking some days off this week, so the release is tonight (thank you Simon for the i18n!)

comment:19 Changed 4 years ago by bastiK

If all the bugs are sorted out, then we can release of course.

comment:20 Changed 4 years ago by Don-vip

I think I finally understand the situation with IE.

IE seems to ignore IP Addresses in SAN and suggest to use a DNS entry instead:
https://connect.microsoft.com/IE/feedback/details/814744/the-ie-doesnt-trust-a-san-certificate-when-connecting-to-ip-address

The issue is (shamely) closed as wontfix. No progress to expect on this side.

Well, no problem, I tried to use a new entry dns:127.0.0.1. It could work... if there wasn't a Java bug that forbids that as well:
https://bugs.openjdk.java.net/browse/JDK-8016345

Maybe we can extend or replace DNSName to remove this check:
http://hg.openjdk.java.net/jdk7u/jdk7u/jdk/file/tip/src/share/classes/sun/security/x509/DNSName.java

Last edited 4 years ago by Don-vip (previous) (diff)

comment:21 Changed 4 years ago by Don-vip

In 7347/josm:

see #10033 - add new entry dns:127.0.0.1 to make it work with IE

comment:22 Changed 4 years ago by Don-vip

So, here's the certificate backend used by each browser per platform, and our status:

Chrome Firefox IE Safari
Windows Windows source:/trunk/images/misc/green_check.png Own NSS source:/trunk/images_nodist/misc/red_x.png Windows source:/trunk/images/misc/green_check.png Windows source:/trunk/images/misc/green_check.png
Linux Shared NSS source:/trunk/images_nodist/misc/red_x.png Own NSS source:/trunk/images_nodist/misc/red_x.png N/A N/A
Mac OSX Keychain source:/trunk/images_nodist/misc/red_x.png Own NSS source:/trunk/images_nodist/misc/red_x.png N/A Keychain source:/trunk/images_nodist/misc/red_x.png

NSS support will likely require a plugin containing NSS and JSS libraries. The Keychain support however must be possible in core.

@Lesath: I have tweaked the certificate to match IE behaviour and it looks like it also pleases Safari on Windows, now. Can you please try again to add this code to PlatformOSX and install the certificate? I think it should allow to make the warning disappear completely:

    @Override
    public boolean setupHttpsCertificate(String entryAlias, KeyStore.TrustedCertificateEntry trustedCert)
            throws KeyStoreException, NoSuchAlgorithmException, CertificateException, IOException {
        KeyStore ks = KeyStore.getInstance("KeychainStore", "Apple");
        ks.load(null, null):
        // Look for certificate to install
        String alias = ks.getCertificateAlias(trustedCert.getTrustedCertificate());
        if (alias != null) {
            // JOSM certificate found, return
            Main.debug(tr("JOSM localhost certificate found in {0} keystore: {1}", tr("Apple Keychain"), alias));
            return false;
        }
        // JOSM certificate not found, warn user
        StringBuilder message = new StringBuilder("<html>");
        message.append(tr("Remote Control is configured to provide HTTPS support.<br>"+
                "This requires to add a custom certificate generated by JOSM to the Apple Keychain.<br><br>"+
                "You are now going to be prompted by OSX to confirm this operation.<br>"+
                "To enable proper HTTPS support, <b>please click Yes</b> in next dialog.<br><br>"+
                "If unsure, you can also click No then disable HTTPS support in Remote Control preferences."));
        message.append("</html>");
        JOptionPane.showMessageDialog(Main.parent, message.toString(), tr("HTTPS support in Remote Control"), JOptionPane.INFORMATION_MESSAGE);
        // install it to Apple Keychain, used Chrome and Safari, but not by Firefox
        Main.info(tr("Adding JOSM localhost certificate to {0} keystore", tr("Apple Keychain")));
        ks.setEntry(entryAlias, trustedCert, null);
        return true;
    }

Can you also tell me if there's any UI warning shown by OSX when installing the certificate (like Windows), or not? If not, the JOSM information message can be removed.

Last edited 3 years ago by skyper (previous) (diff)

comment:23 Changed 4 years ago by naoliv

Cc: naoliv added

comment:25 Changed 4 years ago by Don-vip

I'm not sure they can do anything from their side. Your issue is likely to be closed.
@Lesath: any news please?

comment:26 Changed 4 years ago by Don-vip

Milestone: 14.0814.09

comment:27 Changed 4 years ago by stoecker

What they could do a osm.org is setup a osm.org https-josm-link, which redirects to a osm.org http-josm-link and this one then calls JOSM, so you get a JumpOutOfTLS function. But I don't know if this is a wise idea at all - Breaks the idea of HTTPS.

comment:28 Changed 4 years ago by Don-vip

Milestone: 14.0914.10

Move complicated/risky tickets to next milestone.

comment:29 Changed 4 years ago by Don-vip

Milestone: 14.1014.11

Not enough time/resources for these tickets this month.

comment:30 Changed 3 years ago by Don-vip

Milestone: 14.11

won't have any time soon for this.

comment:31 Changed 3 years ago by Stereo

Cc: Stereo added

comment:32 Changed 3 years ago by Don-vip

Ticket #11170 has been marked as a duplicate of this ticket.

comment:33 Changed 3 years ago by james2432

Seems to be a browser safety more than an OS problem, I have installed the cert in JOSM(by clicking on the install button)
The browser when browsing to ​https://127.0.0.1:8112/load_and_zoom?left=8.19&right=8.20&top=48.605&bottom=48.590&select=node413602999 gave me a security exception page in firefox. I have to accept the security exception (adds the cert to the firefox certificates) and the links now work.

comment:34 Changed 14 months ago by Stereo

#14397 has another idea for macOS if ticket:10033#comment:22 doesn't work.

@Don-vip, I could test your code if you can create a build for me?

comment:35 Changed 14 months ago by Stereo

https://stackoverflow.com/questions/24333480/how-to-add-a-trusted-certificate-autority-to-firefox-with-jss has code to add certificates to the Firefox NSS. I can also test that :)

comment:36 Changed 14 months ago by Stereo

Ticket #14397 has been marked as a duplicate of this ticket.

comment:37 Changed 10 months ago by Don-vip

In 12458/josm:

see #10033 - remove workaround for IE (not needed anymore for IE11 on Windows 10, works also for Edge)

comment:38 Changed 4 months ago by nicolas17

Since Firefox 55 (ticket) and Chrome 53 (ticket), an HTTPS page loading http://127.0.0.1:port/ will work and will not give a mixed-content error. Note this only works for the IP address literals 127.0.0.1 and ::1, not for localhost.

Let's stop messing with self-signed certs.

comment:39 Changed 4 months ago by Stereo

That's great news nicolas17, thanks!

This leaves us only with Safari on macOS then, although I would imagine that Safari users would tend to be iD users. I've tried the latest Safari, 11.0.2, and it indeed blocks e.g. <iframe src="http://127.0.0.1:8111/version" /> with "[blocked] The page at about:blank was not allowed to display insecure content from http://127.0.0.1:8111/version."

comment:40 Changed 4 months ago by nicolas17

There is also a draft spec for CORS and RFC1918 which would add a variant of CORS for any requests going from public webpages to private addresses (like localhost), even if they are plain old GETs, even from an <img> or <iframe>.

As far as I know, no browser implements this yet. But when/if browsers implement it, then 1. JOSM needs to handle the OPTIONS preflight request and return the correct Access-Control-Allow-* headers, and 2. the website doing the request must be HTTPS.

comment:41 Changed 4 months ago by Don-vip

Great news! We should be able to completely drop https support then, and it should work out of the box.

comment:42 Changed 4 months ago by Stereo

Please re-open, Safari on macOS still needs this, unfortunately. And does IE/Edge not need https still?

comment:44 in reply to:  41 ; Changed 4 months ago by stoecker

Replying to Don-vip:

We should be able to completely drop https support then, and it should work out of the box.

There is no reason for this. Https support may have other reasons/uses, e.g. non-localhost access.

comment:45 Changed 4 months ago by Stereo

I've never heard of anyone running josm non-locally you? The certificate wouldn't match anyway, and openstreetmap.org calls localhost, not an arbitrary host. If someone is willing to go through the trouble to fiddle with the josm certificate store, they're more than willing to nc -l 8111 | nc josm.example.com 8111

comment:46 in reply to:  44 Changed 4 months ago by Don-vip

Replying to stoecker:

Https support may have other reasons/uses, e.g. non-localhost access.

I don't think it works, as we issue the certificate for "localhost". Even if it worked, there would be little added value of using https on a local network for this, as there is no credentials or personal information exchanged.

What I see however is the burden of maintaining this feature. We have a lot of platform-dependent code, use of internal Sun APIs, Java 9 command line arguments...

Changed 4 months ago by Don-vip

what it looks like if we remove HTTP support for remote control

comment:47 Changed 4 months ago by bastiK

Props to Vincent for making it work in the first place, but I would very much welcome the application of this patch...

comment:48 Changed 4 months ago by Stereo

https://github.com/openstreetmap/openstreetmap-website/pull/1707 has some browser statistics - probably the closest we can get to reality.

Can anyone make https://stereo.lu/cors.html load in IE 11 while JOSM is running without getting warnings, maybe by adding http://127.0.0.1:8111 to the trusted sites?

Last edited 4 months ago by Stereo (previous) (diff)

comment:49 Changed 4 months ago by Don-vip

Milestone: 18.05

OK so we have only about 40% of visitors using a compatible browser. Let's wait some months to see how the situation evolves:

  • Next version of Edge will be released in March (with Windows 10 1803)
  • Firefox ESR 60 will be released in May
  • Safari: no idea, but there is at least a new version every year

@Stereo: don't waste your time for IE11 now. We'll see when the new version of Edge is released if IE11 benefits from this feature (IE11 and Edge share some code on Windows 10)

comment:50 Changed 4 months ago by stoecker

what it looks like if we remove HTTP support for remote control

Hmm, I still don't like dropping HTTPS support much. I'm currently in the process of implementing TLS in several tools and dropping support sounds like the wrong way to me. Wouldn't dropping all the auto-gen-stuff and replacing it by loading a cert/key file instead also remove all strange dependencies?

comment:51 Changed 4 months ago by Don-vip

Yes it would remove sun imports and java 9 command line arguments, and become an expert option.

comment:52 Changed 3 months ago by Don-vip

Some (bad) news:

  • WebKit surprised me by closing their bug as Wontfix. So Safari will never work: macOS users will have to use another browser for this feature if Webkit team persists to not follow a W3C standard, unlike all other browsers.
  • OSM operation team suddenly switched to https only. So the feature does not work anymore for a lot of users I guess. This PR must be merged asap.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as new The owner will remain team.
as The resolution will be set.
to The owner will be changed from team to the specified user.
The owner will change to anonymous
as duplicate The resolution will be set to duplicate.The specified ticket will be cross-referenced with this ticket
The owner will be changed from team to anonymous.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.