Modify

Opened 11 months ago

Closed 11 months ago

Last modified 11 months ago

#15547 closed defect (fixed)

Serious autocomplete performance degradation

Reported by: SafwatHalaby Owned by: Don-vip
Priority: major Milestone: 17.11
Component: Core Version:
Keywords: regression performance Cc:

Description (last modified by Klumbumbus)

Intro

I deal with a huge dataset (about 30k bus stops nodes) almost daily.
This was never a problem before, but since recently, JOSM often hangs when I edit the bus stop tags.
Waiting for a few seconds (can be up to ~30) always resolves this.

It happens seems to happen when editing massively used keys. For instance, all stops have a
"ref" key, and when I click "edit" on a single stop's "ref", JOSM
stalls. A similar issue occurs when adding "description" to changeset
uploads. Description is also used by all stops. So it appears this is triggered whenever auto-complete is in use.

Profiling

"I can reproduce this locally on JOSM 13106. According to jvisualvm cpu
profiling (on Oracle JDK), most time is spent in method
org.openstreetmap.josm.data.tagging.ac.AutoCompletionSet.add"
-- mmd

It's a CPU starvation issue and not a memory overuse issue.
(Core at 100% for 10-30 seconds).

Test case

Run the Overpass query below to obtain a huge bus stop dataset, then try editing the ref or name of a stop.
12712 works flawlessly.
13053 hangs for 10-30 seconds.

Query

[out:xml][timeout:90][bbox:29.4013195,33.8818359,33.4131022,36.0791016]
;
(
area(3601473946); // Israel
area(3601803010); // Judea and Samaria district
)->.a;
(
  node["highway"="bus_stop"](area.a);
  way["highway"="bus_stop"](area.a);
)->.b;
(rel(bw.b);rel(bn.b))->.routes;
(.b;way(bn.b);)->.b;
(.b;node(w.b);)->.b;
(.b;.routes;);
out meta;

Status report

URL:http://josm.openstreetmap.de/svn/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2017-09-02 14:47:18 +0200 (Sat, 02 Sep 2017)
Build-Date:2017-09-02 20:53:16
Revision:12712
Relative:URL: ^/trunk

Identification: JOSM/1.5 (12712 en) Linux Debian GNU/Linux 9.2 (stretch)
Memory Usage: 468 MB / 811 MB (227 MB allocated, but free)
Java version: 1.8.0_141-8u141-b15-1~deb9u1-b15, Oracle Corporation, OpenJDK 64-Bit Server VM
Screen: :0.0 1366x768
Maximum Screen Size: 1366x768
Java package: openjdk-8-jre:amd64-8u141-b15-1~deb9u1
Java ATK Wrapper package: libatk-wrapper-java:all-0.33.3-13
Dataset consistency test: No problems found

Nothing fancy installed, everything is default. No extra plugins, validator rules, map paint styles, presets, etc.

Attachments (0)

Change History (11)

comment:1 Changed 11 months ago by Klumbumbus

Description: modified (diff)
Keywords: regression performance added

comment:2 Changed 11 months ago by Don-vip

Milestone: 17.11
Owner: changed from team to Don-vip
Status: newassigned

comment:3 Changed 11 months ago by Don-vip

Resolution: fixed
Status: assignedclosed

In 13121/josm:

fix #15547 - fix major performance drawback of AutoCompletionSet (regression from r12859)

comment:4 Changed 11 months ago by Don-vip

I think it's OK now but I have not tested it much against potential side effects. Can you please tell me if you find anything unusual in autocompletion tomorrow?

comment:5 Changed 11 months ago by mmd

I did a very quick check on r13121, performance issue seems to be solved.

comment:6 Changed 11 months ago by SafwatHalaby

Thank you for the quick response. I will test this when I have the time.

For the record, the issue was originally posted and discussed here: https://lists.openstreetmap.org/pipermail/talk/2017-November/079535.html

There is also a separate discussion which was started up at around the same time, but it also talks about memory errors. I don't know whether or not the issues are related:

https://forum.openstreetmap.org/viewtopic.php?id=60403

comment:7 Changed 11 months ago by anonymous

13124 didn't solve the issue.

URL:http://josm.openstreetmap.de/svn/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2017-11-15 20:09:06 +0100 (Wed, 15 Nov 2017)
Build-Date:2017-11-16 02:31:57
Revision:13124
Relative:URL: ^/trunk

Identification: JOSM/1.5 (13124 es) Linux openSUSE Tumbleweed
Memory Usage: 435 MB / 3545 MB (73 MB allocated, but free)
Java version: 1.8.0_144-b01, Oracle Corporation, OpenJDK 64-Bit Server VM
Screen: :0.0 1920x1080
Maximum Screen Size: 1920x1080
Java package: java-1_8_0-openjdk:x86_64-1.8.0.144
Dataset consistency test: No problems found

Plugins:
+ Create_grid_of_ways (32699)
+ ElevationProfile (33815)
+ FixAddresses (33718)
+ HouseNumberTaggingTool (33714)
+ ImproveOsm (122)
+ ImproveWay (12)
+ PicLayer (33822)
+ apache-commons (33668)
+ conflation (0.5.5)
+ contourmerge (1032)
+ dataimport (33581)
+ editgpx (33782)
+ ejml (32680)
+ geotools (33380)
+ graphview (33004)
+ imagery_offset_db (33774)
+ jts (32699)
+ junctionchecking (33778)
+ lakewalker (33712)
+ log4j (32699)
+ measurement (33760)
+ merge-overlap (33436)
+ michigan_left (33004)
+ opendata (33791)
+ pbf (33568)
+ pdfimport (33579)
+ public_transport (33817)
+ reverter (33572)
+ routing (33794)
+ turnlanes (33294)
+ turnrestrictions (33780)
+ undelete (33835)
+ utilsplugin2 (33742)

Tagging presets:
+ http://somewhere.dk/email/josm/source.xml
+ https://josm.openstreetmap.de/josmfile?page=Presets/Quick-highways&zip=1

Map paint styles:
- https://josm.openstreetmap.de/josmfile?page=Styles/Maxspeed&style&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Noname&style&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Surface&style&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Surface-DataEntry&style&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Styles/Lane_and_Road_Attributes&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/BesideTheRoad_Speed&zip=1

Last errors/warnings:
- W: Actualizar complementos - org.openstreetmap.josm.plugins.PluginHandler$UpdatePluginsMessagePanel[,0,0,0x0,invalid,layout=java.awt.GridBagLayout,alignmentX=0.0,alignmentY=0.0,border=,flags=9,maximumSize=,minimumSize=,preferredSize=]
- W: Invalid jar file ''<josm.pref>/plugins/PicLayer.jar.new'' (exists: false, canRead: false)
- W: Invalid jar file ''<josm.pref>/plugins/ElevationProfile.jar.new'' (exists: false, canRead: false)
- W: Invalid jar file ''<josm.pref>/plugins/undelete.jar.new'' (exists: false, canRead: false)
- W: Invalid jar file ''<josm.pref>/plugins/public_transport.jar.new'' (exists: false, canRead: false)
- W: No configuration settings found.  Using hardcoded default values for all pools.

...Deleted...

comment:8 Changed 11 months ago by Don-vip

Who's speaking?

comment:9 Changed 11 months ago by SafwatHalaby

I don't know. It was not me. I have not tested this yet.

comment:10 Changed 11 months ago by Don-vip

OK. Sometimes we have random anonymous comments on totally different subjects. Unless we get concrete information, I will only consider your feedback ;) When will you be able to test it?

comment:11 Changed 11 months ago by SafwatHalaby

Tested with 13150.
I could not reproduce the original "Test case" described in my first post. There were no noticeable hangs.
Thank you!

Last edited 11 months ago by SafwatHalaby (previous) (diff)

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Don-vip.
as The resolution will be set.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.