Modify

Opened 2 months ago

Closed 2 months ago

Last modified 5 weeks ago

#14649 closed defect (fixed)

[Patch] Cannot access HTTPS Dutch WMTS servers without using WebStart or Linux

Reported by: Allroads Owned by: team
Priority: major Milestone: 17.04
Component: Core imagery Version: latest
Keywords: wmts certificate Netherlands quovadis jnlp webstart https windows Cc: stoecker

Description

Hello,

This wmts url is working in the webstart. Tested by others.
http://geodata.nationaalgeoregister.nl/tiles/service/wmts/bgtachtergrond?SERVICE=WMTS&request=GetCapabilities

Projection set on Rijksdriehoek RD EPSG:28992 Important!!

Used in webstart it works.
With JOSM .exe .jar, it is not.

Imagery: Also in the available default entries

https://josm.openstreetmap.de/wiki/Maps/Netherlands#PDOKLuchtfotoBeeldmateriaal25cmWMTS

This ortho wmts is not working in Josm .jar and with webstart it is.

It does not matter if service=WMS or set on service=WMTS in the wmts string both does not work.

Josm 11826
Java Version 8 Update 121 1.8.0_121-b13 latest
JOSM is on: No proxy

Started Josm with .bat file
cmd.exe copy past field started error wmts.

2017-04-14 14:20:54.268 INFO: GET https://geodata.nationaalgeoregister.nl/luchtfoto/wmts?&request=GetCapabilities&service=WMS -> !!!
2017-04-14 14:20:54.268 WARNING: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target. Cause: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target. Cause: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at sun.security.ssl.Alerts.getSSLException(Unknown Source)
        at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)
        at sun.security.ssl.Handshaker.fatalSE(Unknown Source)
        at sun.security.ssl.Handshaker.fatalSE(Unknown Source)
        at sun.security.ssl.ClientHandshaker.serverCertificate(Unknown Source)
        at sun.security.ssl.ClientHandshaker.processMessage(Unknown Source)
        at sun.security.ssl.Handshaker.processLoop(Unknown Source)
        at sun.security.ssl.Handshaker.process_record(Unknown Source)
        at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
        at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
        at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
        at sun.net.www.protocol.https.HttpsClient.afterConnect(Unknown Source)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(Unknown Source)
        at org.openstreetmap.josm.tools.HttpClient.connect(HttpClient.java:125)
        at org.openstreetmap.josm.tools.HttpClient.connect(HttpClient.java:75)
        at org.openstreetmap.josm.io.CachedFile.checkLocal(CachedFile.java:472)
        at org.openstreetmap.josm.io.CachedFile.getFile(CachedFile.java:272)
        at org.openstreetmap.josm.io.CachedFile.getInputStream(CachedFile.java:207)
        at org.openstreetmap.josm.data.imagery.WMTSTileSource.getCapabilities(WMTSTileSource.java:331)
        at org.openstreetmap.josm.data.imagery.WMTSTileSource.<init>(WMTSTileSource.java:278)
        at org.openstreetmap.josm.actions.AddImageryLayerAction.convertImagery(AddImageryLayerAction.java:99)
        at org.openstreetmap.josm.actions.AddImageryLayerAction.actionPerformed(AddImageryLayerAction.java:138)
        at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
        at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
        at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
        at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
        at javax.swing.AbstractButton.doClick(Unknown Source)
        at javax.swing.plaf.basic.BasicMenuItemUI.doClick(Unknown Source)
        at javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(Unknown Source)
        at java.awt.AWTEventMulticaster.mouseReleased(Unknown Source)
        at java.awt.Component.processMouseEvent(Unknown Source)
        at javax.swing.JComponent.processMouseEvent(Unknown Source)
        at java.awt.Component.processEvent(Unknown Source)
        at java.awt.Container.processEvent(Unknown Source)
        at java.awt.Component.dispatchEventImpl(Unknown Source)
        at java.awt.Container.dispatchEventImpl(Unknown Source)
        at java.awt.Component.dispatchEvent(Unknown Source)
        at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
        at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
        at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
        at java.awt.Container.dispatchEventImpl(Unknown Source)
        at java.awt.Window.dispatchEventImpl(Unknown Source)
        at java.awt.Component.dispatchEvent(Unknown Source)
        at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
        at java.awt.EventQueue.access$500(Unknown Source)
        at java.awt.EventQueue$3.run(Unknown Source)
        at java.awt.EventQueue$3.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
        at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
        at java.awt.EventQueue$4.run(Unknown Source)
        at java.awt.EventQueue$4.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
        at java.awt.EventQueue.dispatchEvent(Unknown Source)
        at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
        at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
        at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
        at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
        at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
        at java.awt.EventDispatchThread.run(Unknown Source)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at sun.security.validator.PKIXValidator.doBuild(Unknown Source)
        at sun.security.validator.PKIXValidator.engineValidate(Unknown Source)
        at sun.security.validator.Validator.validate(Unknown Source)
        at sun.security.ssl.X509TrustManagerImpl.validate(Unknown Source)
        at sun.security.ssl.X509TrustManagerImpl.checkTrusted(Unknown Source)
        at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(Unknown Source)
        ... 59 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at sun.security.provider.certpath.SunCertPathBuilder.build(Unknown Source)
        at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(Unknown Source)
        at java.security.cert.CertPathBuilder.build(Unknown Source)
        ... 65 more


Attachments (2)

windows_jre1.8.121.txt (15.8 KB) - added by Don-vip 2 months ago.
14649.diff (10.6 KB) - added by Don-vip 2 months ago.

Download all attachments as: .zip

Change History (29)

comment:1 Changed 2 months ago by bastiK

Please add full status report!

comment:2 Changed 2 months ago by Allroads

Please provide any additional information below. Attach a screenshot if possible.

URL:http://josm.openstreetmap.de/svn/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2017-04-02 01:45:00 +0200 (Sun, 02 Apr 2017)
Build-Date:2017-04-02 01:34:50
Revision:11826
Relative:URL: ^/trunk

Identification: JOSM/1.5 (11826 en) Windows 10 64-Bit
Memory Usage: 497 MB / 910 MB (341 MB allocated, but free)
Java version: 1.8.0_121-b13, Oracle Corporation, Java HotSpot(TM) 64-Bit Server VM
Screen: \Display0 1680x1050, \Display1 1680x1050
Maximum Screen Size: 1680x1050

Plugins:
deleted privat

Tagging presets:
deleted privat

Map paint styles:
deleted privat

Last errors/warnings:
- E: Failed to locate image ''
- W:  Privat: Could not get presets icon 
- E: Failed to locate image ''
- W:  Privat: Could not get presets icon 
- E: Failed to locate image ''
- W:  Privat: Could not get presets icon 
- W: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target. Cause: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target. Cause: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
- E: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target. Cause: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target. Cause: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

comment:3 Changed 2 months ago by bastiK

Please update imagery list and try again. It is important that the capabilities request URL starts with http:// and not https://.

@team: The root certificate is available on Debian, but apparently not in the Java keystore on Windows:

$ keytool -keystore /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/security/cacerts -list
[...]
debian:staat_der_nederlanden_root_ca_-_g3.pem, Feb 7, 2016, trustedCertEntry, 
Certificate fingerprint (SHA1): D8:EB:6B:41:51:92:59:E0:F3:E7:85:00:C0:3D:B6:88:97:C9:EE:FC
[...]

Changed 2 months ago by Don-vip

Attachment: windows_jre1.8.121.txt added

comment:4 in reply to:  3 Changed 2 months ago by Don-vip

Replying to bastiK:

@team: The root certificate is available on Debian, but apparently not in the Java keystore on Windows:

Confirmed, see attached file.

comment:5 Changed 2 months ago by Allroads

But how is this solved.

Is it done with a new JOSM update?

When?

comment:6 Changed 2 months ago by Don-vip

Cc: stoecker added
Keywords: certificate Netherlands quovadis added

There are 6 QuoVadis entries in the Java keystore, but not "Staat der Nederlanden Root". QuoVadis roots have been updated recently, see javabug:8145955 but they only include the "standard" root CAs.

Here the CA is "QuoVadis CSP - PKI Overheid CA" which seems to be a system specifi to Dutch government.

Should we include it in JOSM?

comment:7 Changed 2 months ago by bastiK

We could do that, but the tile URL template that is returned by the capabilities request makes JOSM request the tiles over plain http:

2017-04-14 16:14:13.754 INFO: GET http://geodata.nationaalgeoregister.nl/luchtfoto/wmts?SERVICE=WMTS&REQUEST=GetTile&VERSION=1.0.0&LAYER=2016_ortho25&STYLE=default&FORMAT=image/png8&tileMatrixSet=OGC:1.0:GoogleMapsCompatible&tileMatrix=11&tileRow=677&tileCol=1057 -> 200 (63.0 kB)

So there isn't gained much by making the initial request over https and then fetching the potentially more sensitive data over http.

comment:8 in reply to:  5 Changed 2 months ago by bastiK

Replying to Allroads:

But how is this solved.

By changing the URL in the imagery database from https to http.

Is it done with a new JOSM update?

No, you just need to update the Imagery list in the preferences or wait 24 h for auto-update.

comment:9 in reply to:  6 ; Changed 2 months ago by stoecker

Replying to Don-vip:

Here the CA is "QuoVadis CSP - PKI Overheid CA" which seems to be a system specifi to Dutch government.

Should we include it in JOSM?

I'd say no:

  • I don't find it in firefox
  • It is a single use case
  • The use is solved by the maps update

comment:10 Changed 2 months ago by bastiK

If any, we should include the root certificate "staat_der_nederlanden_root_ca_-_g3.pem" (which is included in Firefox and Chrome) and not "QuoVadis CSP - PKI Overheid CA".

comment:11 in reply to:  9 ; Changed 2 months ago by Allroads

Replying to stoecker:

Replying to Don-vip:

I'd say no:

  • I don't find it in firefox
  • It is a single use case

single use, I ask on behalve of the whole Dutch community. Not for me, only person.

  • The use is solved by the maps update

Not only for the default entries, like ortho photo.

But there many other layers to use licence PD or CC0 1.0
All together in a big list
https://geodata.nationaalgeoregister.nl/tiles/service/wmts?SERVICE=WMTS&request=GetCapabilities

When Government uses https links for use, metadata, I find that it must be easy as possible to add new wmts entries for everyone. They give in the above link, that should be working directly with https url.
Not everyone knows the http s story, so they are annoyed, thinking the layers do not work.
This gives questions on forum and to the Government. Do we want that?

It happend also to me. Why does the layer do not work.

This is also with other wms layers of the Goverment.

What good certificates are, I do not know, it is not my knowledge.

comment:12 in reply to:  11 Changed 2 months ago by stoecker

Replying to Allroads:

  • It is a single use case

single use, I ask on behalve of the whole Dutch community. Not for me, only person.

There is a single user of these certificates, the Dutch government.

Not everyone knows the http s story, so they are annoyed, thinking the layers do not work.

Which would be correct. They do not work reliable with https, as the certificates aren't included in Java (and probably other tools as well).

This gives questions on forum and to the Government. Do we want that?

It was their decision to choose these certificates.

What good certificates are, I do not know, it is not my knowledge.

That's simple - they must be trustworthy enough to be accepted in as many products as possible and they must be included in these products.

We added own certificate storage in JOSM for the cases were Java is behind all the other browsers. But the rule is we do not add anything, which the big browsers don't add and there must be a high enough demand to include a certificate. If there are other possibilities not requiring to include a certificate these should be used.

For now JOSM included the CA's providing free certificates, as these enable normal people to offer https services, so it is in our interest to support these (now only Let's Encrypt) even if Java does not (yet). To support a single government CA I see no real benefit for JOSM.

comment:13 Changed 2 months ago by Don-vip

Java does support Let's Encrypt now, since 8u101 released in July 2016.

Looking at usage statistics, 82.5% of our users use a compatible version:

J        649 ( 5.7%) Java/1.8.0_101
J        120 ( 1.1%) Java/1.8.0_102
J       1124 ( 9.9%) Java/1.8.0_111
J        103 ( 0.9%) Java/1.8.0_112
J       7366 (64.9%) Java/1.8.0_121

And 17% do not:

J          9 ( 0.1%) Java/1.8.0
J         17 ( 0.1%) Java/1.8.0_05
J          9 ( 0.1%) Java/1.8.0_11
J         22 ( 0.2%) Java/1.8.0_20
J        122 ( 1.1%) Java/1.8.0_25
J        144 ( 1.3%) Java/1.8.0_31
J         56 ( 0.5%) Java/1.8.0_40
J        126 ( 1.1%) Java/1.8.0_45
J         81 ( 0.7%) Java/1.8.0_51
J        129 ( 1.1%) Java/1.8.0_60
J         96 ( 0.8%) Java/1.8.0_65
J        246 ( 2.2%) Java/1.8.0_66
J         52 ( 0.5%) Java/1.8.0_71
J          6 ( 0.1%) Java/1.8.0_72
J         97 ( 0.9%) Java/1.8.0_73
J         41 ( 0.4%) Java/1.8.0_74
J        141 ( 1.2%) Java/1.8.0_77
J        461 ( 4.1%) Java/1.8.0_91
J         62 ( 0.5%) Java/1.8.0_92

We will soon be able to remove it as well, see #14652.

Last edited 2 months ago by Don-vip (previous) (diff)

comment:14 Changed 2 months ago by Don-vip

Back to the point.
The certificate is trusted by (at least) Mozilla, Microsoft and Debian.
Plus, Allroads raises a very good point: the capabilities returned by the HTTP link returns a lot of services in HTTPS.
This seems a nice enhancement to Dutch community to be able to add https maps out of the box, without the need to edit the JOSM wiki.

It's not so different from the inclusion of NTv2 grids which only concerned a few countries.

comment:15 Changed 2 months ago by Allroads

What happend, I open a Openstreetmap forum topic, because a men of the Dutch Government said, that the webstart JOSM did show the layers with https. Therefor the question here.
https://forum.openstreetmap.org/viewtopic.php?id=58039 Dutch.
Now several of us did install the certificate in the JAVA keystore. I exported them from Firefox.
Because we wanted to know, if it works.
https is working and we get the tiles. All good.
You can read how in the forumlink.
StaatderNederlandenRootCA-G2.crt
but also
StaatderNederlandenOrganisatieCA-G2.crt
QuoVadisCSP-PKIOverheidCA-G2.crt

Not everyone, who tag in The Netherlands, who uses JOSM read the Dutch forum. Topic is going down in the list.
The manual solution is not for everyone.

It is not only the layers, who are written in the wiki, the default ones.
Many of us, use/to test, single layers as well and manually give in the wms/wmts.
Copied, nowadays, from metadata the htpps link.
Knowing the s problem, I forget it also, not getting a layer list.
Ouch, thinking I knew it.

Last edited 2 months ago by Allroads (previous) (diff)

comment:16 in reply to:  14 Changed 2 months ago by stoecker

Replying to Don-vip:

Back to the point.
The certificate is trusted by (at least) Mozilla, Microsoft and Debian.

Which one? "QuoVadis CSP - PKI Overheid CA" is not in my list. I found one "Staat der Nederlanden Root CA G2" thought.

comment:17 Changed 2 months ago by Don-vip

I was speaking of the root CA, which is "Staat der Nederlanden Root CA G2".

This is also strange that Java, when running with WebStart, accesses the local Windows keystore (which contains this CA) while a standard run with java -jar does not. Hard to understand for end-users.

comment:18 Changed 2 months ago by Allroads

One user wrote:
After 11826 installed, JOSM did not gave a pop up anymore with failures but the wmts layer bgtstandaard ( topic start url deep zoom in for layer) was not shown, on screen where tiles should be, in red "Error: Problem loading tile"

The version before 11826, had this popup report.

And
Before that the wmts layer worked.

Last edited 2 months ago by Allroads (previous) (diff)

comment:19 Changed 2 months ago by Allroads

When I installed certificate
First I installed only this one.
StaatderNederlandenRootCA-G2.crt
Then tested it and I did get the tiles on the screen.

Then installed the other ones. Maybe not needed.

comment:20 Changed 2 months ago by Don-vip

Keywords: jnlp webstart https added
Summary: WMTS: webstart working, with JOSM .jar start, faillure wmts.Cannot access HTTPS Dutch WMTS servers without using WebStart or Linux

comment:21 Changed 2 months ago by Don-vip

Summary: Cannot access HTTPS Dutch WMTS servers without using WebStart or Linux[Patch] Cannot access HTTPS Dutch WMTS servers without using WebStart or Linux

Better solution that should satisfy everyone: load the certificate from Windows keystore if not present in Java keystore. Nothing has to be embedded, just the alias and hash in source code, see attached patch (tested OK).

@Dirk: is it OK for you?

Changed 2 months ago by Don-vip

Attachment: 14649.diff added

comment:22 Changed 2 months ago by stoecker

Looks like a useful solution.

comment:23 Changed 2 months ago by Don-vip

Milestone: 17.04

comment:24 Changed 2 months ago by Don-vip

Resolution: fixed
Status: newclosed

In 11943/josm:

fix #14649 - load Dutch Government (G2 & G3) certificates from Windows keystore if not found in Java keystore

comment:25 Changed 2 months ago by Don-vip

In 11944/josm:

see #14649 - fix @since

comment:26 Changed 2 months ago by Don-vip

Keywords: windows added

comment:27 Changed 5 weeks ago by Don-vip

In 12241/josm:

see #11924, see #14649 - java 9 does not seem to include Dutch certificates yet, load them from /usr/share/ca-certificates/mozilla (see Debian ca-certificates package)

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain team.
as The resolution will be set. Next status will be 'closed'.
The resolution will be deleted. Next status will be 'reopened'.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.