Mapillary plugin: terrible slowdown

[richlv]

Not sure how this could be reproduced, but my scenario was:

1. Dwonload some area, (manually) download Mapillary images.
2. Set filter "not older than one year".
3. In another area, download Mapillary images again.

These steps might have nothing to do with the slowdown reason, though.

JOSM got terribly slow, and Mapillary images (the green circles) appeared one by one about once per second.

I've seen this happen a few times before, but only in the past few weeks - don't recall seeing it before.

URL:https://josm.openstreetmap.de/svn/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2020-12-28 22:03:23 +0100 (Mon, 28 Dec 2020)
Build-Date:2020-12-30 02:30:55
Revision:17428
Relative:URL: ^/trunk

Identification: JOSM/1.5 (17428 en_GB) Mac OS X 10.15.7
OS Build number: Mac OS X 10.15.7 (19H114)
Memory Usage: 1511 MB / 1820 MB (802 MB allocated, but free)
Java version: 1.8.0_271-b09, Oracle Corporation, Java HotSpot(TM) 64-Bit Server VM
Look and Feel: com.apple.laf.AquaLookAndFeel
Screen: Display 1127230987 1920×1080 (scaling 1.00×1.00) Display 69733382 1680×1050 (scaling 1.00×1.00)
Maximum Screen Size: 1920×1080
Best cursor sizes: 16×16→16×16, 32×32→32×32
VM arguments: [-Djnlp.application.href=https://josm.openstreetmap.de/download/josm.jnlp, -Djava.util.Arrays.useLegacyMergeSort=true, -Djnlp.tk=awt, -Djnlpx.jvm=<java.home>/bin/java, -Djnlpx.splashport=-1, -Djnlpx.home=<java.home>/bin, -Djnlpx.remove=false, -Djnlpx.offline=false, -Djnlpx.relaunch=true, -Djnlpx.session.data=/var/folders/nl/flqxqsmj5q963r7tcnfrdt3c0000gn/T/session6054319306617707269, -Djnlpx.heapsize=-1,2147483648, -Djava.security.policy=file:<java.home>/lib/security/javaws.policy, -DtrustProxy=true, -Djnlpx.origFilenameArg=/Users/richlv/Library/Application Support/Oracle/Java/Deployment/cache/6.0/56/1ee8cfb8-72e8e992, -Dsun.awt.warmup=true, -Djava.security.manager]
Dataset consistency test: No problems found

Plugins:
+ InfoMode (35543)
+ Mapillary (1.5.32)
+ PicLayer (2a9aa7a)
+ apache-commons (35524)
+ apache-http (35589)
+ ejml (35458)
+ geotools (35458)
+ imagery_offset_db (35640)
+ javafx-osx (35655)
+ jaxb (35543)
+ jna (35662)
+ jts (35458)
+ measurement (35640)
+ opendata (35640)
+ pbf (35650)
+ photo_geotagging (35640)
+ reverter (35640)
+ utilsplugin2 (35674)

Map paint styles:
- /Users/richlv/Desktop/ChangeFontSize.mapcss
- https://josm.openstreetmap.de/josmfile?page=Styles/Coloured_Streets&zip=1

Last errors/warnings:
- 10977.068 W: An image is not painted, because it is null or has no LatLon!
- 10978.990 W: An image is not painted, because it is null or has no LatLon!
- 10979.576 W: An image is not painted, because it is null or has no LatLon!
- 10981.097 W: An image is not painted, because it is null or has no LatLon!
- 10981.113 W: An image is not painted, because it is null or has no LatLon!
- 10981.631 W: An image is not painted, because it is null or has no LatLon!
- 10981.641 W: An image is not painted, because it is null or has no LatLon!
- 10983.619 W: An image is not painted, because it is null or has no LatLon!
- 10984.151 W: An image is not painted, because it is null or has no LatLon!
- 10984.179 W: An image is not painted, because it is null or has no LatLon!

[richlv]

If I hide Mapillary layer, JOSM becomes notably faster. Re-enabling Mapillary layer makes it crawl again.
One-by-one appearing images in sequences continue appearing like that.

[taylor.smock]

Quick question: Past week (after December 24) or prior to that.

I probably ought to look into using a QuadBucket for Mapillary (this should drastically improve performance with many images).

I need to make some changes in JOSM core first though. See #18434 (which I forgot about due to time constraints).

[richlv]

I do not explicitly recall noticing this before Dec 24th, but it is also possible I did not hit the exact conditions to trigger this.

[taylor.smock]

Replying to richlv:

I do not explicitly recall noticing this before Dec 24th, but it is also possible I did not hit the exact conditions to trigger this.

You also (recently) started filing bugs and looking for stuff like that.
I suspect that it is just a long running issue (although initial performance while everything is being downloaded has degraded).

[GhostFoxSledgehammer]

When trying to reproduce #20349 I downloaded a rather large area and during the third phase of download (detections) my CPU was being used at 100% which felt like "terrible slowdown". In the log I could only see detections being downloaded one after another. This could also be the root cause for #18242.

Edit: This was when a single download was running, with multiple downloads its even worse.

[richlv]

Replying to taylor.smock:

You also (recently) started filing bugs and looking for stuff like that.
I suspect that it is just a long running issue (although initial performance while everything is being downloaded has degraded).

True, but I've been using the Mapillary plugin for quite some time, reports are in large part motivated by your great attention to them ;)

[taylor.smock]

Lower thread priority

[taylor.smock]

I just uploaded a jar file that you can use to test to see if it alleviates the responsiveness issue. It will still (initially) have high CPU usage on download, but the thread priority has been set lower (on the scale of 1 to 10, it was a 4 -- I've set it to 2 in attachment:Mapillary.jar​ for testing).

[taylor.smock]

@GhostFoxSledgehammer: attachment:20331.lower_priority.patch​ has the change made to attachment:Mapillary.jar​, if you want to play around with the settings on your machine.

[GhostFoxSledgehammer]

Replying to taylor.smock:

@GhostFoxSledgehammer: attachment:20331.lower_priority.patch​ has the change made to attachment:Mapillary.jar​, if you want to play around with the settings on your machine.

Like you said, CPU usage is still high but its not as aggressive as before, so GUI is still responsive and background music doesn't crack.

[taylor.smock]

OK. I've merged in the alleviation in ​8dfd66a3cfdcf0045139d9d99557e2d830d1dc59.

[taylor.smock]

I've hopefully fixed this in ​v1.5.35. Unfortunately, due to another fix, the fix for this isn't relevant anymore.