Add Overpass download dialog to core, provide Overpass Turbo wizard

[simon04]

This patch adds the Overpass download dialog to core, and in addition provides Overpass Turbo wizard (using Java's script engine) to quickly and intuitively generate an Overpass Query …

[bastiK]

Layout is strange, see attachment.

[bastiK]

It may backfire if we have too many hardcoded URLs in JOSM-core:

https://overpass-api.de/api/

We already have a meta-list of overpass servers: MirroredDownloadInfo.

https://raw.githubusercontent.com/tyrasd/overpass-turbo/b0ef5ebbdd353c1bc7e45da527cab03498b4cbef/

Maybe mirror this code at a place that we have control over.

[simon04]

bastiK, thank you for your review.

• The layout should be fixed (damn GBC.fill() vs. GBC.fill(GBC.HORIZONTAL) (maybe we should provide GBC.fillHorizontal() and deprecate the GBC.fill()?)
• Hardcoded URLs: This feature doesn't seem to be used extensively since the page hasn't been edited for 3 years. Is it worth the effort? At least, I made the URL configurable in the advanced settings.
• Linking to overpass-turbo: Since the commit SHA is specified, the code shouldn't change overtime (unless the repository is removed completely or a heavy git rebase is performed. The question for me is whether we a core feature should require a download of 4 JavaScript files upon first usage?

[simon04]

Like for #11755, I submitted a few upstream pull-requests to ease the integration of the JavaScript library ​https://github.com/tyrasd/overpass-turbo/pull/189, ​https://github.com/tyrasd/overpass-turbo/pull/191, ​https://github.com/tyrasd/overpass-turbo/pull/192

We end up with a 41K (not yet zipped) JavaScript file which can be bundled with JOSM. So the fetching of four js files from github.com upon first usage is now gone. Plus, we have two test cases. :)

[simon04]

In 8684/josm:

see #11428 - Add Overpass download dialog to core, provide Overpass Turbo wizard

[simon04]

Adapted mirrored_download plugin: [o31520], [o31521].

[Don-vip]

In 8685/josm:

see #11428 - fix checkstyle issues

[Klumbumbus]

How do I use this feature? I don't have such a box like in the screenshot of comment:2

I use latest r8685.

[simon04]

You would use File → Download from Overpass API (this had been provided by the mirrored_download plugin before [o31520]). Then you would see a dialog like attachment:2015-08-24-093410_998x581_scrot.png​​

[Don-vip]

Is the mirrored download plugin still useful by the way?

[Klumbumbus]

I tried the railway=tram example and I get the following error:

IO Exception

Failed to upload data to or download data from 'https://overpass-api.de/api/interpreter?data=%5Bbbox%5D%5Btimeout%3A25%5D%3B%0A%2F%2F+gather+results%0A%28%0A++%2F%2F+query+part+for%3A+%E2%80%9Crailway%3Dtram%E2%80%9D%0A++node%5B%22railway%22%3D%22tram%22%5D%3B%0A++way%5B%22railway%22%3D%22tram%22%5D%3B%0A++relation%5B%22railway%22%3D%22tram%22%5D%3B%0A%29%3B%0A%2F%2F+print+results%0Aout+meta%3B%0A%3E%3B%0Aout+meta%3B&bbox=12.8289413,50.6851478,12.8708267,50.7164607' due to a problem with transferring data. Details (untranslated): handshake alert: unrecognized_name

from the console:

INFO: GET https://overpass-api.de/api/interpreter?data=%5Bbbox%5D%5Btimeout%3A25%5D%3B%0A%2F%2F+gather+results%0A%28%0A++%2F%2F+query+part+for%3A+%E2%80%9Crailway%3Dtram%E2%80%9D%0A++node%5B%22railway%22%3D%22tram%22%5D%3B%0A++way%5B%22railway%22%3D%22tram%22%5D%3B%0A++relation%5B%22railway%22%3D%22tram%22%5D%3B%0A%29%3B%0A%2F%2F+print+results%0Aout+meta%3B%0A%3E%3B%0Aout+meta%3B&bbox=12.9183769,50.8297945,12.92799,50.8353239
ERROR: javax.net.ssl.SSLProtocolException: handshake alert:  unrecognized_name
javax.net.ssl.SSLProtocolException: handshake alert:  unrecognized_name
	at sun.security.ssl.ClientHandshaker.handshakeAlert(Unknown Source)
	at sun.security.ssl.SSLSocketImpl.recvAlert(Unknown Source)
	at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
	at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
	at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
	at sun.net.www.protocol.https.HttpsClient.afterConnect(Unknown Source)
	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(Unknown Source)
	at org.openstreetmap.josm.io.OsmServerReader.getInputStreamRaw(OsmServerReader.java:158)
	at org.openstreetmap.josm.io.OsmServerReader.getInputStreamRaw(OsmServerReader.java:104)
	at org.openstreetmap.josm.io.OsmServerReader.getInputStream(OsmServerReader.java:68)
	at org.openstreetmap.josm.io.OsmServerReader.getInputStream(OsmServerReader.java:51)
	at org.openstreetmap.josm.io.BoundingBoxDownloader.parseOsm(BoundingBoxDownloader.java:147)
	at org.openstreetmap.josm.actions.OverpassDownloadAction$OverpassDownloadReader.parseOsm(OverpassDownloadAction.java:216)
	at org.openstreetmap.josm.actions.downloadtasks.DownloadOsmTask$DownloadTask.parseDataSet(DownloadOsmTask.java:306)
	at org.openstreetmap.josm.actions.downloadtasks.DownloadOsmTask$DownloadTask.realRun(DownloadOsmTask.java:314)
	at org.openstreetmap.josm.gui.PleaseWaitRunnable.doRealRun(PleaseWaitRunnable.java:93)
	at org.openstreetmap.josm.gui.PleaseWaitRunnable.run(PleaseWaitRunnable.java:161)
	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)
ERROR: org.openstreetmap.josm.io.OsmTransferException: Could not connect to the OSM server. Please check your internet connection.. Cause: javax.net.ssl.SSLProtocolException: handshake alert:  unrecognized_name
org.openstreetmap.josm.io.OsmTransferException: Could not connect to the OSM server. Please check your internet connection.
	at org.openstreetmap.josm.io.OsmServerReader.getInputStreamRaw(OsmServerReader.java:161)
	at org.openstreetmap.josm.io.OsmServerReader.getInputStreamRaw(OsmServerReader.java:104)
	at org.openstreetmap.josm.io.OsmServerReader.getInputStream(OsmServerReader.java:68)
	at org.openstreetmap.josm.io.OsmServerReader.getInputStream(OsmServerReader.java:51)
	at org.openstreetmap.josm.io.BoundingBoxDownloader.parseOsm(BoundingBoxDownloader.java:147)
	at org.openstreetmap.josm.actions.OverpassDownloadAction$OverpassDownloadReader.parseOsm(OverpassDownloadAction.java:216)
	at org.openstreetmap.josm.actions.downloadtasks.DownloadOsmTask$DownloadTask.parseDataSet(DownloadOsmTask.java:306)
	at org.openstreetmap.josm.actions.downloadtasks.DownloadOsmTask$DownloadTask.realRun(DownloadOsmTask.java:314)
	at org.openstreetmap.josm.gui.PleaseWaitRunnable.doRealRun(PleaseWaitRunnable.java:93)
	at org.openstreetmap.josm.gui.PleaseWaitRunnable.run(PleaseWaitRunnable.java:161)
	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)
Caused by: javax.net.ssl.SSLProtocolException: handshake alert:  unrecognized_name
	at sun.security.ssl.ClientHandshaker.handshakeAlert(Unknown Source)
	at sun.security.ssl.SSLSocketImpl.recvAlert(Unknown Source)
	at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
	at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
	at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
	at sun.net.www.protocol.https.HttpsClient.afterConnect(Unknown Source)
	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(Unknown Source)
	at org.openstreetmap.josm.io.OsmServerReader.getInputStreamRaw(OsmServerReader.java:158)
	... 14 more

[simon04]

@Klumbumbus: Uff, Java (on Windows?) does not like the StarCom SSL certificate. Please try setting download.overpass.url to http://overpass-api.de/api/. Do we need to stick to HTTP due to this certificate issue?

@Don-vip: The plugin (1) features XAPI queries (which is somehow superseded by the overpass turbo query wizard), (2) a UI for changing the Overpass server, and (3) the "mirror mode" (downloading the unfiltered data from the Overpass server). (1) could be dropped perhaps, (3) should be easy to add and I don't know about (2).

[Klumbumbus]

Replying to simon04:

Please try setting download.overpass.url to http://overpass-api.de/api/.

This works for me.

[naoliv]

Could we have some kind of history in this dialog?
For example, I would like to see some of my last queries stored somewhere.

[naoliv]

Another could: could we also have an option to select the overpass server?
For example, right now ​https://overpass-api.de/api/ is giving timeout while ​http://overpass.osm.rambler.ru/cgi/ works.

[simon04]

In 8688/josm:

see #11428 - Make Overpass server configurable

[simon04]

Replying to simon04:

@Klumbumbus: Uff, Java (on Windows?) does not like the StarCom SSL certificate. Please try setting download.overpass.url to http://overpass-api.de/api/. Do we need to stick to HTTP due to this certificate issue?

r8688 defaults to HTTP to avoid this certificate issue.

@Don-vip: The plugin (1) features XAPI queries (which is somehow superseded by the overpass turbo query wizard), (2) a UI for changing the Overpass server, and (3) the "mirror mode" (downloading the unfiltered data from the Overpass server). (1) could be dropped perhaps, (3) should be easy to add and I don't know about (2).

(3) is already implemented (one has to empty the query field). (2) has come with r8688.

Can we deprecate the mirrored_download plugin?

[simon04]

In 8689/josm:

see #11428 - Make Overpass download an expert action

[simon04]

In 8711/josm:

see #11428 - Overpass Download: use Monospace font for query

[simon04]

In [o31534:31535]:

JOSM/mirrored_download: Drop shortcut in favour of JOSM core action

[simon04]

In 8712/josm:

see #11428 - Overpass Download: take nice shortcut from mirrored_download plugin

[simon04]

In 8713/josm:

see #11428 - Overpass Download: make help work

[simon04]

In 8716/josm:

see #11428 - Overpass Download: use JosmTextArea for monospace font

[skyper]

See also #7647 and #7670.

[simon04]

Replying to naoliv:

Could we have some kind of history in this dialog?
For example, I would like to see some of my last queries stored somewhere.

See #10023.

[simon04]

In 8744/josm:

see #11428 - Refactoring (class/package structure)

[simon04]

In 8745/josm:

see #11428 - Deprecate mirrored_download plugin which is part of core

[simon04]

In [o31543]:

JOSM/mirrored_download: Remove mirrored_download plugin (has been integrated to JOSM core in 2015-08, see #josm11428)

[simon04]

:-)

[simon04]

In 8782/josm:

see #11428 - Extract Overpass request errors

Those error messages are nicely displayed in the error dialog.

This is a workaround for ​https://github.com/drolbr/Overpass-API/issues/94

[simon04]

Example for syntax error reporting

[simon04]

In 8787/josm:

see #11428 - Change Overpass task name in progress monitor to "Contacting Server..."

[mmd]

@Don-vip : as a solution outside of Overpass API is now in place, there's nothing left to do in ​https://github.com/drolbr/Overpass-API/issues/94

Could you please close your GitHub ticket?

Thanks

[baditaflorin]

Replying to simon04:

In 8745/josm:

see #11428 - Deprecate mirrored_download plugin which is part of core

Where exactly can you find the mirrored_download plugin ?\

The beauty of mirrored_download is that you can search by only saying

[highway=*]

Now i can only find Overpass Api , where i have to start defining things ... i don`t have a list of exception ...

[simon04]

It has been removed in favour of the query wizard, see comment:15. You can input highway=* in the query wizard field and obtain an Overpass query.