[patch - needs rework] Enable Overpass D/L / Add Overpass Filter Tab to DownloadDialog

[cmuelle8]

This is a first attempt to give josm users direct access to Overpass.
The tab is only added to the DownloadDialog if JOSM Expert mode is used.
Most things are self explanatory and if you can't figure out how to use the dialog, the patch probably isn't for you, yet.

I invite ppl to hack at it, if they like it. Atm I don't have the time to support this further, so if someone takes the code from here and e.g. adds a better filter selection using the

• existing TaggingPresets dialog (problem: depends on an existing DataSet in the editor) or
• takes the preset menu + dialogs to deduce filter values

that's great.

Enjoy,
cmuelle8

[cmuelle8]

add an Overpass DownloadSelection to JOSM

[bastiK]

Hi, are you aware of the mirrored download plugin? I think it provides similar features.

[cmuelle8]

Replying to bastiK:

Hi, are you aware of the mirrored download plugin? I think it provides similar features.

I was not. After checking it, I think it is not as useful. It uses XAPI compatibility layer. Furthermore it hides most of the URL from the user, so you can't just copy+paste URLs with the dialog. I also don't like the GUI integration at two spots in the menu, instead of the single tab extension in the download dialog.

greetings

[bastiK]

Replying to cmuelle8:

Replying to bastiK:

Hi, are you aware of the mirrored download plugin? I think it provides similar features.

I was not. After checking it, I think it is not as useful.

Anyway, please try to include your features into the existing plugin.

It uses XAPI compatibility layer. Furthermore it hides most of the URL from the user, so you can't just copy+paste URLs with the dialog.

I think both the XAPI compatibility and the more advanced query languages are useful. What about a button "Advanced query", that leads to another modal dialog with more options?

I also don't like the GUI integration at two spots in the menu, instead of the single tab extension in the download dialog.

I think a second menu item is appropriate. The Overpass API and the main OSM API are two different services and the user should be aware of that.

[cmuelle8]

Replying to bastiK:

Anyway, please try to include your features into the existing plugin.

This won't happen any time soon, I guess.

It uses XAPI compatibility layer. Furthermore it hides most of the URL from the user, so you can't just copy+paste URLs with the dialog.

I think both the XAPI compatibility and the more advanced query languages are useful. What about a button "Advanced query", that leads to another modal dialog with more options?

No, you do not need XAPI at all. The way my dialog is designed, users of XAPI are immediately familiar with how to use Overpass QL. They can make a transient switch, because using just the filter combobox is essentially the same (read identical) to using XAPI.

The difference is, that users who are aware of the flexibility of Overpass QL can simply make a more complex query (and have it remembered by the historycombobox) by using the URL textfield.

I also don't like the GUI integration at two spots in the menu, instead of the single tab extension in the download dialog.

I think a second menu item is appropriate. The Overpass API and the main OSM API are two different services and the user should be aware of that.

I agree in the point that the user should be aware. But he is due to the fact

• the tab is named
• that the URL is shown the data originates from

As this is an advanced feature and only shown in "expert" mode, I do not see a reason to make this even more obvious.

That said, I do not suggest merging this as is. As of now isValidURL() and isValidFilter() need a clear definition in the code. Essentially, the idea is, to only use Overpass if the user "leaves" the DownloadDialog with the "Overpass Filter" tab shown. For this - isValidFilter() still needs to be adjusted. As long as this is not done, I agree that it's not obvious enough, even to an expert user which d/l service is used.

gn8

[bastiK]

Replying to cmuelle8:

Replying to bastiK:

Anyway, please try to include your features into the existing plugin.

This won't happen any time soon, I guess.

We clearly don't need two interfaces to the Overpass API, please understand. The plugin by Roland (the Overpass author) was first, so it is the one that stays. In my opinion this feature is better placed in a plugin anyway, because the service is relatively new. However, you are free to significantly extend the existing plugin and rework it to your linking, provided Roland is Ok with your changes.

It uses XAPI compatibility layer. Furthermore it hides most of the URL from the user, so you can't just copy+paste URLs with the dialog.

I think both the XAPI compatibility and the more advanced query languages are useful. What about a button "Advanced query", that leads to another modal dialog with more options?

No, you do not need XAPI at all. The way my dialog is designed, users of XAPI are immediately familiar with how to use Overpass QL. They can make a transient switch, because using just the filter combobox is essentially the same (read identical) to using XAPI.

Be aware that the Overpass QL is not easy to learn and understand, so maybe the full request need not be displayed all the time. Also it is more readable when broken over multiple lines.

Best, Paul

[cmuelle8]

Replying to bastiK:

We clearly don't need two interfaces to the Overpass API, please understand. The plugin by Roland (the Overpass author) was first, so it is the one that stays.

I guess Roland is busy with a lot of other things and the reason he wrote this originally back in February was to fill the void - just as was my intention. If JOSM-development has switched to using what was first, instead of using what's best, I'll troll myself. You don't need rev-control if the maxime is sticking to "what was first"..

IMHO you should just wait for Roland's opinion on the matter and then decide further.

greetings

[stoecker]

JOSM always used the "What was first" rule. We simply don't drop existing code, but always improve it. So the correct way would be to improve the existing plugin. I agree with Paul that this is no core feature and should remain in plugin form. So any changes should be relative to the plugin code and not the core. The core even has a plugin feature to register downloader-tabs, so this is no problem at all.

[bastiK]

Let me put it this way: There are two code bases, your patch and the plugin. Both have aspects that make it better than the other:

• Plugin downloads the list of URLs from the JOSM wiki (​http://josm.openstreetmap.de/wiki/MirroredDownloadInfo) in order to react to a change in server configuration, patch uses a hardcoded list.
• Patch has drop down for server selection right in the download dialog which is better IMHO than a separate entry in the main menu, as done in the plugin.
• Patch allows entering of complex Overpass QL, plugin lacks this at the moment.
• Plugin is well tested, patch still needs a little polishing.

What this boils down to is that both works have to be merged.

[roland.olbricht]

Replying to cmuelle8:

IMHO you should just wait for Roland's opinion on the matter and then decide further.

I agree with Paul. The Overpass download facility is not yet a core feature of OSM. And because JOSM shall depend on as few external services as possible, this functionality is better placed in a plugin.

One can think about making more advertising for plugins in general and this specific one to avoid double work on the same issue, but that is clearly a separate topic.

Concerning the question of whether this should be one or two menu items: Assume there were another plugin that somehow extends the download dialogue. If the mirrored_download plugin replaces the menu entry, then it would be impossible to use both plugins at the same JOSM instance. Thus, it is a rather good idea to have it in a separate menu entry.

Concerning the Copy&Paste of URLs: a complete URL can be pasted into the "Download from source ..." dialogue, so I don't see why it is necessary to also do that in the plugin. But in the other direction, to copy the composed URL to use it elsewhere, makes sense. I think to show the URL that JOSM finally sends in a non-editable TextField could be a good solution.

Please keep in mind some use cases I was asked for:

• Somebody without XAPI or Overpass QL knowledge wants to get a faster download. This user should not be bothered with configuring any details.
• Somebody wants to download from his private Overpass (or jXAPI) server and wants to configure this server permanently.

The desire to download directly Overpass QL queries is of course also an important desire.

Personally, I would love to end with a design, where the first tab shows only the interactive map, as the standard download window shows.

An extra tab can then contain server configuration, and another tab the space for formulating queries, maybe with distinct tabs for Overpass QL and XAPI. This fulfills the above mentioned requirements as follows:

• Who doesn't want to care about the details can simply ignore the extra tabs.
• The guy with his private server needs to configure the server in the respective tab only once and forever
• The separate tab for Overpass QL does contain enough space for multi-line queries, the display of the URL and so on

At this point of thinking one may reconsider to simply add the tabs into the mainstream download dialogue. This would be most elegant and eleminate the second menu item, but I don't know whether this is technically possible without screwing up other plugins. Paul, what do you think?

For a schedule. I put time at the moment primarly in a mobile client and the augmented diffs. Thus, I'm sorry, but I can't do significant work on this plugin in this August. So, cmuelle8, you and anybody else are invited to rework the plugin yourself.

[stoecker]

At this point of thinking one may reconsider to simply add the tabs into the mainstream download dialogue. This would be most elegant and eleminate the second menu item, but I don't know whether this is technically possible without screwing up other plugins. Paul, what do you think?

As said that is possible. There is a special API for this which is unused since I integrated slippy_map_chooser in core a long time ago :-)

[bastiK]

Replying to roland.olbricht:

Personally, I would love to end with a design, where the first tab shows only the interactive map, as the standard download window shows.

An extra tab can then contain server configuration, and another tab the space for formulating queries, maybe with distinct tabs for Overpass QL and XAPI. This fulfills the above mentioned requirements as follows:

• Who doesn't want to care about the details can simply ignore the extra tabs.
• The guy with his private server needs to configure the server in the respective tab only once and forever
• The separate tab for Overpass QL does contain enough space for multi-line queries, the display of the URL and so on

At this point of thinking one may reconsider to simply add the tabs into the mainstream download dialogue. This would be most elegant and eleminate the second menu item,

Sounds good in general. But please keep in mind that at the moment, the tabs are consistently used for bbox selection, only. All other options are visible, no matter which tab is currently selected. An alternative would be a second tab bar on the top of the dialog. Or we could add buttons, that launch another window.

but I don't know whether this is technically possible without screwing up other plugins. Paul, what do you think?

Certainly technically possible. If there are problems, we deal with them.

For a schedule. I put time at the moment primarly in a mobile client and the augmented diffs.

Glad to hear the augmented diffs make progress. This will make a lot of people happy, I'm sure of it. :)

[cmuelle8]

A minimal effort way to have this, is pluging in the overpass-turbo web frontend into a download dialog tab. 'special API' as mentioned might be used for this to do this within a josm plugin, whether this be mirrored_download or any other.

This also benefits from future improvements done to overpass-turbo.de, rather than reimplementing lots of the same stuff in java code.

Currently, if overpass-turbo.de is visited using a web browser, the website offers to export queried data to josm. For this to work it uses the remote-control interface and josm has to be configured to listen on a local port for this data.

If the website runs embedded within a download dialog tab, queried data should be transferable directly by josm-api calls, instead of depending on the availability of the remote-control plugin.

[simon04]

mirrored_download is now part of core (see #11428)

[ThiagoPv]

Now that this feature was implemented into core the original idea of this ticket can be implemented, putting Overpass download option inside DownloadDialog as a new tab. It would make much more sense than how it is now.

[simon04]

Does it? The current tabs in the normal download dialog are all for selecting a bbox for the download. The to-be-added Overpass tab would be a filter on the objects within the bbox. Also it might not be obvious at all that an Overpass query has been specified and one might be confused why not all objects are being retrieved.

[anonymous]

Replying to simon04:

Does it? The current tabs in the normal download dialog are all for selecting a bbox for the download. The to-be-added Overpass tab would be a filter on the objects within the bbox. Also it might not be obvious at all that an Overpass query has been specified and one might be confused why not all objects are being retrieved.

I thought a little over your comment, analyzed those windows dialogs and got a new possible approach for this subject. The best place for Overpass filter is inside the already existing tabs in download dialog, as an expert option hided by default and fired by a button or text on the upper right or bottom left that will show a side panel with query fields and maybe other options. It will consider the bbox for limiting search and will have a button to ignore the bbox if setted. It will be perfect because it will bring the same feeling of the Overpass Turbo website. This ease would make much more people start usig this incredible feature because it will be much more accessible.
I don't believe this method would make people gets confused with unintentional filters because it will require an intentional action of the user to make use of it.

This idea fled a little from the original idea of this ticket.
I'm sorry any language issue, english isn't my natural language.

[cmuelle8]

implemented meanwhile