Modify

Opened 8 months ago

Closed 7 weeks ago

Last modified 14 hours ago

#18164 closed enhancement (fixed)

Migrate OverpassTurboQueryWizard to Java or something else

Reported by: simon04 Owned by: simon04
Priority: normal Milestone: 20.05
Component: Core Version:
Keywords: overpass javascript nashorn sotm19 java15 Cc: Don-vip, tyr_asd

Description

Currently, the query wizard is based on JavaScript and executed using java's Nashorn engine. To get rid of the already deprecated Nashorn dependency, we should implement the query wizard in Java.

Within overpass-turbo, the parsing is implemented in PEG.js, see the file https://github.com/tyrasd/overpass-turbo/blob/master/js/ffs/ffs.pegjs

Attachments (1)

18164.patch (12.1 KB) - added by simon04 3 months ago.

Download all attachments as: .zip

Change History (31)

comment:1 Changed 8 months ago by simon04

In 15370/josm:

see #18164 - Extend OverpassTurboQueryWizardTest

Based on tests from https://github.com/tyrasd/overpass-turbo/blob/master/tests/test.ffs.js

comment:2 Changed 8 months ago by Don-vip

Keywords: sotm19 added

comment:3 Changed 3 months ago by simon04

Keywords: java15 added

comment:4 Changed 3 months ago by simon04

Cc: Don-vip added
Milestone: 20.03

Hosted overpass-wizard? · Issue #11 · https://github.com/tyrasd/overpass-wizard/issues/11

comment:5 Changed 3 months ago by simon04

Summary: Migrate OverpassTurboQueryWizard to JavaMigrate OverpassTurboQueryWizard to Java or something else

comment:6 Changed 3 months ago by simon04

Implement overpass-wizard-server by simon04 · Pull Request #12 · https://github.com/tyrasd/overpass-wizard/pull/12

Running instance on my computer: https://overpass-wizard.josm.eu/wizard/?search=amenity=drinking_water&comment=false

@Don-vip, can we deploy a Node.js application on the JOSM server? Here are the instructions: https://github.com/tyrasd/overpass-wizard/tree/master/server

Last edited 3 months ago by simon04 (previous) (diff)

comment:7 Changed 3 months ago by simon04

Cc: tyr_asd added

comment:8 Changed 3 months ago by simon04

Owner: changed from team to simon04
Status: newassigned

Changed 3 months ago by simon04

Attachment: 18164.patch added

comment:9 Changed 3 months ago by simon04

Next step: (a) deploy overpass-wizard-server, or (b) setup https://josm.openstreetmap.de/remote/overpass-wizard/ to ​https://overpass-wizard.josm.eu/wizard/

comment:10 in reply to:  6 ; Changed 3 months ago by Don-vip

Replying to simon04:

@Don-vip, can we deploy a Node.js application on the JOSM server? Here are the instructions: https://github.com/tyrasd/overpass-wizard/tree/master/server

Sure, I can do that. But do we really need it? If I understand the ticket correctly, only https://github.com/tyrasd/overpass-turbo/blob/master/js/ffs/ffs.pegjs would need to be migrated to Java? It seems doable, no?

comment:12 Changed 3 months ago by Don-vip

So around 800 lines of JavaScript. I'll give a try if you don't beat me to it :) I'd like to keep it simple and Java-only :)

comment:13 Changed 3 months ago by simon04

A different approach we could investigate: sparse the query using your SearchCompiler` and turn the parsed AST into an Overpass Query. Advantages: not yet another parser for key=value; reused code; same syntax as used for search within JOSM; disadvantages: likely a slightly different syntax than accepted by overpass-wizard.
Vincent, what do you think?

comment:14 in reply to:  10 ; Changed 3 months ago by tyr_asd

The most up to date wizard code would be at ​https://github.com/tyrasd/overpass-wizard/. If you think about porting it to Java, probably everything should be ported (also tests, documentation, etc.). It's probably not too hard to do, but keep in mind that some queries might involve geocoder loopups (e.g. "tourism=hotel in Heidelberg"). Also, the auto-generated parser (https://github.com/tyrasd/overpass-wizard/blob/master/parser.js – currently about 1500 lines of code) would need to be ported as well (and again every time the base grammar is modified), if you don't want to port pegjs to java as well.

comment:15 in reply to:  13 Changed 3 months ago by tyr_asd

Replying to simon04:

A different approach we could investigate: sparse the query using your SearchCompiler` and turn the parsed AST into an Overpass Query. Advantages: not yet another parser for key=value; reused code; same syntax as used for search within JOSM; disadvantages: likely a slightly different syntax than accepted by overpass-wizard.
Vincent, what do you think?

Here, one potential issue comes to my mind: Probably not all features of the JOSM search are possible to translate to an Overpass Query easily. And some features of the wizard syntax are not yet present in the search functionality (e.g. "… in Heidelberg" to search in a given region). While these are easy to translate into an Overpass query, these features would not be easy to implement on the JOSM search itself. So you would probably end up with two different dialects of a query syntax again. ¯\_(ツ)_/¯

comment:16 in reply to:  14 Changed 3 months ago by Don-vip

Replying to tyr_asd:

Also, the auto-generated parser (https://github.com/tyrasd/overpass-wizard/blob/master/parser.js – currently about 1500 lines of code) would need to be ported as well (and again every time the base grammar is modified), if you don't want to port pegjs to java as well.

I guess we can translate the pegjs parser definition using JavaCC. The parser itself would be generated like MapCSS.

comment:17 Changed 3 months ago by tyr_asd

Ah, yes. That sounds like a better approach. I would be interested to see how you would translate the grammar of the parser to JavaCC. :)
Out of curiousity: You didn't use JavaCC for the "search compiler", right? Is that just because of legacy reasons, or did you encounter problems when defining a grammar for it?

comment:18 in reply to:  17 Changed 3 months ago by simon04

Replying to tyr_asd:

Out of curiousity: You didn't use JavaCC for the "search compiler", right? Is that just because of legacy reasons, or did you encounter problems when defining a grammar for it?

The SearchCompiler has been introduced in 2008 via r513. We started using JavaCC in 2013 for MapCSS (introduced in r3848).

comment:19 Changed 2 months ago by Don-vip

Milestone: 20.0320.04

comment:20 Changed 7 weeks ago by simon04

In 16261/josm:

see #18164 - Extend OverpassTurboQueryWizardTest

comment:21 Changed 7 weeks ago by simon04

Resolution: fixed
Status: assignedclosed

In 16262/josm:

fix #18164 - Migrate OverpassTurboQueryWizard to Java

The new OverpassTurboQueryWizard first invokes SearchCompiler, and then turns the AST into an Overpass QL.

comment:22 Changed 7 weeks ago by GerdP

This is not supported by JOSM, but the wizard at http://overpass-turbo.eu/ converts it:
type:relation and type=restriction and ~"restr.*"~"no.*"

comment:23 Changed 7 weeks ago by simon04

In 16264/josm:

see #18164 - OverpassTurboQueryWizard: fix "newer" statements

comment:24 Changed 7 weeks ago by simon04

In 16336/josm:

fix #19112, see #18164 - OverpassTurboQueryWizard: use OsmPrimitiveType.getAPIName

comment:25 Changed 6 weeks ago by simon04

In 16354/josm:

see #18164 - OverpassQueryWizardDialog: reuse SearchDialog

comment:26 Changed 6 weeks ago by simon04

In 16355/josm:

see #18164 - OverpassQueryWizard: add icon

Icon taken from https://publicdomainvectors.org/en/free-clipart/Wand-with-Stars-Vector-Graphics/8217.html (PD and CC0 licensed)

comment:27 Changed 6 weeks ago by simon04

In 16358/josm:

see #18164 - Rename class to SearchCompilerQueryWizard

comment:28 Changed 6 weeks ago by simon04

In 16359/josm:

see #18164 - Rename Overpass query wizard in dialogs

comment:29 Changed 3 weeks ago by Klumbumbus

Milestone: 20.0420.05

Milestone renamed

comment:30 Changed 14 hours ago by Don-vip

Nice, thanks a lot Simon for having taken care of this subject!

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain simon04.
as The resolution will be set.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.