extremely slow UI - pedestrian crossing markings

[mnalis]

What steps will reproduce the problem?

1. select a node and use F3 to select Highways/Waypoints/Pedestrian Crossing
2. notice that it takes somewhat longer for dialog to come up than for other presets (about 3 seconds; versus sub-second for other POI presets)
3. try to change markings - note that each click takes extremely long to process (about 6 seconds; versus sub-second for other POI presets, and even other drop drown lists in that same preset)

What is the expected result?

it should be as fast as others (sub-second)

What happens instead?

it takes half a dozen seconds for click to register!

Please provide any additional information below. Attach a screenshot if possible.

Attached is a video. 00:00-00:16 is normally working preset for reference (bump gate), 00:17-01:10 is slow Pedestrian crossing.
Slowness seems to vary depending on amount of data loaded, but that preset is always slower than all the others that I have tried.

Relative:URL: ^/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2024-07-02 17:10:50 +0200 (Tue, 02 Jul 2024)
Revision:19128
Build-Date:2024-07-03 01:31:15
URL:https://josm.openstreetmap.de/svn/trunk

Identification: JOSM/1.5 (19128 en) Linux Debian GNU/Linux 12 (bookworm)
Memory Usage: 1528 MB / 2987 MB (245 MB allocated, but free)
Java version: 17.0.11+9-Debian-1deb12u1, Debian, OpenJDK 64-Bit Server VM
Look and Feel: javax.swing.plaf.metal.MetalLookAndFeel
Screen: :0.0 1920x1080x[Multi depth]@60Hz (scaling 1.00×1.00)
Maximum Screen Size: 1920×1080
Best cursor sizes: 16×16→16×16, 32×32→32×32
Environment variable LANG: hr_HR.UTF-8
System property file.encoding: UTF-8
System property sun.jnu.encoding: UTF-8
Locale info: en_US
Numbers with default locale: 1234567890 -> 1234567890
Desktop environment: ICEWM
Java package: openjdk-17-jre:amd64-17.0.11+9-1~deb12u1
fonts-noto: fonts-noto:all-20201225-1
VM arguments: [-Dsun.java2d.opengl=true, --add-exports=java.base/sun.security.action=ALL-UNNAMED, --add-exports=java.desktop/com.sun.imageio.plugins.jpeg=ALL-UNNAMED, --add-exports=java.desktop/com.sun.imageio.spi=ALL-UNNAMED]
Program arguments: [2023-08-01_15-43_Tue zadar walk.gpx, 2023-08-01_17-47_Tue walk zadar restoran.gpx, 2023-08-01_18-55_Tue walk plaza Zadar.gpx, 2023-08-02_08-47_Wed walk Zadar Spar.gpx, 2023-08-02_10-32_Wed kombijem po orbitrack.gpx, 2023-08-02_12-42_Wed walk Zadar rucak.gpx, 2023-08-02_16-06_Wed ksenija kombi autobus Zadar.gpx, 2023-08-02_17-41_Wed bus2 zadar.gpx, 2023-08-02_19-10_Wed back hostel.gpx, 2023-08-03_08-32_Thu zadar plaza-rucak.gpx, 2023-08-03_17-44_Thu vecera i Spar Zadar.gpx, 2023-08-04_07-23_Fri dorucak-rucak Zadar.gpx, 2023-08-04_17-36_Fri vecera nevrijeme.gpx, 2023-08-05_07-29_Sat zadar sub.gpx, 2023-08-05_17-28_Sat cafe Donat.gpx, 2023-08-06_07-11_Sun obilazak Zadra.gpx, 2023-08-06_12-49_Sun rucak Zadar.gpx, 2023-08-06_17-44_Sun plaza.gpx, 2023-08-07_07-23_Mon kisa sa plaze.gpx, 2023-08-07_13-44_Mon sa rucka.gpx, 2023-08-07_15-27_Mon bus odlazni.gpx, 2023-08-07_17-55_Mon sa vecere na plazu.gpx, 2023-08-07_21-13_Mon Zadar.gpx, 2023-08-08_07-58_Tue walk povratak u hostel.gpx]
Dataset consistency test: No problems found

Plugins:
+ DirectUpload (36178)
+ FixAddresses (36258)
+ ImageWayPoint (36226)
+ PicLayer (1.0.3)
+ Review.Changes (1.1.1)
+ apache-commons (36273)
+ apache-http (36273)
+ areaselector (2.6.2)
+ austriaaddresshelper (88)
+ buildings_tools (36226)
+ conflation (0.6.11)
+ ejml (36176)
+ imagery_offset_db (36289)
+ jna (36273)
+ jts (36004)
+ log4j (36273)
+ osmwiki-dataitem (0.0.8)
+ pt_assistant (637)
+ reverter (36256)
+ turnrestrictions (36226)
+ undelete (36226)
+ utilsplugin2 (36241)
+ wikipedia (605)

Tagging presets:
+ https://<user.name>.com/prezentacije/OpenStreetMap/josm-preset.xml
+ https://raw.githubusercontent.com/OpenNauticalChart/josm/master/INT-1-preset.xml
+ https://josm.openstreetmap.de/josmfile?page=Presets/Healthcare&zip=1
+ https://raw.githubusercontent.com/osmlab/name-suggestion-index/main/dist/presets/nsi-josm-presets.min.xml

Map paint styles:
- https://josm.openstreetmap.de/josmfile?page=Styles/PublicTransport&zip=1
+ http://bezdomni.net/Styles_HighlightedCapesBays-style.mapcss
+ https://josm.openstreetmap.de/josmfile?page=Styles/ColorWays&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Coloured_Streets&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Modified&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Styles/CyclewaysStylingBNA&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Styles/SidewalksPlus&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/BuildingColors&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Styles/FixmeAndNote&zip=1

Validator rules:
+ https://josm.openstreetmap.de/josmfile?page=Rules/KeepRight&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Rules/QAToolInspiredValidations&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Rules/OSMLint&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Rules/MissingWeekdays&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Rules/SourceObjectTag&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Rules/IndoorRules&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Rules/Pictures&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Rules/MissingOpeningHours&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Rules/CheckDate&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Rules/FacebookWebsite&zip=1

Last errors/warnings:
- 00008.044 E: unable to find dependency log4j for plugin areaselector

[mnalis]

Slow pedestrian crossing preset UI

[taylor.smock]

Java version: 17.0.11+9-Debian-1deb12u1, Debian, OpenJDK 64-Bit Server VM

Since you are on Linux, I hope you don't mind helping me troubleshoot; I'm not seeing anything like that on my machine (mac).

For this, I think ​async profiler will work.

I always profile with both CPU and memory allocations (if there are a lot of memory allocations, one of the JVM internal methods will usually have to spend a lot of time cleaning up).

Once you have downloaded and installed async profiler, you'll want to run the following:

# jps will print process ids (PIDs) of Java processes. Look for `MainApplication`, `Boot`, or `josm`.rm
$ jps
# `-e cpu,alloc,lock` will record CPU, memory allocations, and thread locks
$ asprof start -e cpu,alloc,lock -f 23825_profile.jfr ${PID}
# Replicate the bug on your machine.
$ asprof stop ${PID}

Upload 23825_profile.jfr here.

[gaben]

I noticed this as well.

[taylor.smock]

As long as someone who can repro can profile the problem and upload a JFR file, I can fix it.

I'm not seeing the problem on my machine, assuming I properly understand the steps to reproduce. Alternatively, I can try to walk you (@gaben) through the process.

[gaben]

Ah got it, it's depending on the dataset size. Just use more data to repro. Probably the preset country gate (what is the name, region lock?) is causing the issue, because it was introduced around the same time.

My pleasure, but what do you mean? Nowadays I don't have much time for OSM, let alone JOSM. I'm just lurking to see if I can help somewhere with minimal effort.

[taylor.smock]

Ah got it, it's depending on the dataset size

This is important information. Thanks for finding that.

My pleasure, but what do you mean?

I was talking about walking you through how I troubleshoot performance problems and find the root cause.

Nowadays I don't have much time for OSM, let alone JOSM.

I'm sorry to hear that, but life does go on.

I'm just lurking to see if I can help somewhere with minimal effort.

Thanks for doing that.

[taylor.smock]

I still wasn't able to repro the slow Highways/Waypoints/Pedestrian Crossing issue, but I was able to discover that the wikipedia plugin is incredibly bad when downloading a lot of bounding boxes.

@gaben: Are you on linux, windows, or mac?

EDIT: It is also pretty bad about moving nodes. I'll fix that and see if either one of you can still repro.

[mnalis]

asprof for adding one pedestrian crossing, and clicking once on "yes" for markings.

[mnalis]

Thanks @taylor.smock, I've now uploaded 23825_profile.jfr, hopefully it helps!

[taylor.smock]

I've been able to repro. Thank you @mnalis.

How to repro:

1. JOSM Preferences
2. Tagging Presets -> enable Run data validator on user input
3. Data validator -> Show informational level.

It looks like we have a problem with a listener being called many times.

[mnalis]

I can confirm that disabling Tagging Presets / Run data validator on user input (and restarting JOSM) makes it run fast again!

3. is probably not needed; at least in my case, Data validator / Show informational level was disabled all the time... (greyed-out Show informational level on upload. was enabled though).

[gaben]

Replying to taylor.smock:

I was talking about walking you through how I troubleshoot performance problems and find the root cause.

I'm in, we can discuss it on Matrix and then create a wiki page or add content somewhere. Maybe next week, are you ok with that?

Yes, you have to have the async validation enabled to make a difference. I forgot this as it's my default setting so to say. Windows, Linux same on both.

Also there is a suspicious duplicate entry in the log:

2024-07-30 08:45:02.695 FINE: Running test Tag checker
2024-07-30 08:45:02.695 FINE: Running test Tag checker
2024-07-30 08:45:02.695 FINE: Adding resource://data/validator/addresses.mapcss to tag checker
2024-07-30 08:45:02.697 FINE: Adding resource://data/validator/combinations.mapcss to tag checker
...

On top of other possible issues, maybe the listener firing the validation twice?

[taylor.smock]

Yes, the validation is getting fired multiple times. I put in a quick counter to see how often, and it looks like it gets called twice.

The remaining problem is that the validator is being called in the UI thread.
EDIT: It is actually being called in the worker thread, but the expensive part is in applying the tag to a new primitive to be sent to the worker thread for validation.

src/org/openstreetmap/josm/gui/tagging/presets/items/MultiSelect.java

@@ -94 +94 @@
         }
         p.add(sp, GBC.eol().fill(GBC.HORIZONTAL)); // NOSONAR
 
-        list.addListSelectionListener(l -> support.fireItemValueModified(this, key, getSelectedItem().value));
+        list.addListSelectionListener(l -> {
+            if (!l.getValueIsAdjusting()) {
+                support.fireItemValueModified(this, key, getSelectedItem().value);
+            }
+        });
         list.setToolTipText(getKeyTooltipText());
         list.applyComponentOrientation(OrientationAction.getValueOrientation(key));
 

[taylor.smock]

In 19160/josm:

Fix #23825: UI appears to freeze when a multi-select option is used

There are two problems fixed here:

1. The ListSelectionListener gets called twice; this is fixed by waiting for the list value to stop changing.
2. The act of applyChangeTags clones any primitive that might be affected by the tag change of the primitive. This can get pretty expensive if relations are involved. This is fixed by cloning the primitive in a different thread.

[taylor.smock]

Replying to gaben:

I'm in, we can discuss it on Matrix and then create a wiki page or add content somewhere. Maybe next week, are you ok with that?

Works for me.

[GerdP]

No idea if this is related. Any idea why unit test OAuthAuthenticationPreferencesPanelTest fails only with Java 11?

[taylor.smock]

I'm pretty certain it is unrelated. From the stack trace for the failing test

java.util.ConcurrentModificationException
	at java.base/java.util.TreeMap$PrivateEntryIterator.nextEntry(TreeMap.java:1208)
	at java.base/java.util.TreeMap$KeyIterator.next(TreeMap.java:1262)
	at java.base/java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1047)
	at java.base/java.util.AbstractCollection.addAll(AbstractCollection.java:351)
	at java.base/java.util.HashSet.<init>(HashSet.java:120)
	at org.openstreetmap.josm.io.auth.JosmPreferencesCredentialAgent.storeOAuthAccessToken(JosmPreferencesCredentialAgent.java:133)
	at org.openstreetmap.josm.io.auth.CredentialsManager.storeOAuthAccessToken(CredentialsManager.java:165)
	at org.openstreetmap.josm.gui.preferences.server.OAuthAuthenticationPreferencesPanelTest.tearDown(OAuthAuthenticationPreferencesPanelTest.java:52)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
[...]
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
[...]
	at java.base/java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:189)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)

Looking at that stack trace, it appears to be an issue with one thread making changes to preferences and another thread iterating through the preferences.