Changed Notes don't always upload when quitting JOSM

[daganzdaanda]

I was commenting and closing quite a few notes recently. When I quit JOSM, the window came up with the reminder "these things should be done before quitting", showing correctly that there were changed notes to upload. I made sure that the "upload" option was checked, and pressed the "do these things before quitting" button. JOSM then quit.

When I checked a bit later, at least some of the notes that I worked on were not submitted and were still open on the website. I can't say for sure if none of the left over notes were uploaded, or if some made it to the server and others did not.

This happened to me at least twice in a few days, after I was working on these changesets:
​https://www.openstreetmap.org/changeset/100451340
​https://www.openstreetmap.org/changeset/100234743

But to make things more difficult, I was not able to reproduce this error with just two test notes.
I have no clue what may have been the problem. Is there a timeout for the communication to the server? If so, maybe with many notes to be updated, the timeout is reached and some notes don't make the cut?

[skyper]

Does not sound nice.

Do we backup notes layers under deleted_layers?

@daganzdaanda:
You can save notes layer to a file. Might be useful when trying to reproduce and as safety if it happens again.

[daganzdaanda]

It happened again, and this time, I saved the notes layer first (thanks for the tip!).
Of 11 closed notes, 6 were not uploaded, 5 were uploaded.
See the attached .osn file for the relevant notes.

[daganzdaanda]

Hm, why can't I add another attachment now?? The first file does not contain all the relevant notes.

[daganzdaanda]

All the notes I worked on.

[skyper]

Would you, please, add/attach you status report, thanks.

---

Thanks for your report, however your ticket is incomplete and therefore not helpful in its current form.

Please add all needed information according to this list:

• The required parts of the Status Report from your JOSM.
  • Please, use Report Bug from Help menu and copy & paste.
• Describe what behaviour you expected.
• Describe what did happen instead.
• Describe if and how the issue is reproducible.
• Add any relevant information like error messages or screenshots.

To ensure that all technical relevant information is contained, create new tickets by clicking in JOSMs Main Menu on Help → Report Bug.

Remember: This is a generic notice so we don't need to write the same stuff again and again. It may only apply in parts to the specific case!

[daganzdaanda]

Relative:URL: ^/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2021-03-17 18:58:40 +0100 (Wed, 17 Mar 2021)
Revision:17580
Build-Date:2021-03-17 17:59:56
URL:https://josm.openstreetmap.de/svn/trunk

Identification: JOSM/1.5 (17580 de) Windows 10 64-Bit
OS Build number: Windows 10 Home 2004 (19041)
Memory Usage: 1145 MB / 5500 MB (552 MB allocated, but free)
Java version: 11.0.6+10, AdoptOpenJDK, OpenJDK 64-Bit Server VM
Look and Feel: com.sun.java.swing.plaf.windows.WindowsLookAndFeel
Screen: \Display0 1920×1080 (scaling 1,00×1,00)
Maximum Screen Size: 1920×1080
Best cursor sizes: 16×16→32×32, 32×32→32×32

Plugins:
+ CustomizePublicTransportStop (35640)
+ FixAddresses (35640)
+ HouseNumberTaggingTool (35640)
+ Mapillary (1.5.37.3)
+ OpeningHoursEditor (35640)
+ PicLayer (2a9aa7a)
+ RoadSigns (35640)
+ SimplifyArea (35640)
+ apache-commons (35524)
+ apache-http (35589)
+ buildings_tools (35669)
+ changeset-viewer (25)
+ continuosDownload (91)
+ ejml (35458)
+ ext_tools (35640)
+ geotools (35458)
+ imagery_offset_db (35640)
+ jaxb (35543)
+ jna (35662)
+ jts (35458)
+ opendata (35640)
+ openqa (0.2.1)
+ pdfimport (35640)
+ photo_geotagging (35715)
+ pt_assistant (2.1.10-80-g7d9bba3)
+ reltoolbox (35640)
+ reverter (35688)
+ scripting (30798)
+ tageditor (35640)
+ terracer (35640)
+ turnlanes (35640)
+ turnlanes-tagging (288)
+ turnrestrictions (35640)
+ undelete (35640)
+ utilsplugin2 (35691)
+ waydownloader (35640)
+ wikipedia (1.1.4)

Tagging presets:
+ https://josm.openstreetmap.de/josmfile?page=Presets/Heritage&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/OpenTollMap&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/OneClick&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/LaneAttributes&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Healthcare&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Community_Centre&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/BuildingPreset&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Crafts&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Historical_Objects&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Leaftype&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/NewTags&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/NewParkingFeatures&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/OpenStationMap&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/ParkingLanes&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/PublicTransportOneClick&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Simple_Indoor_Tagging&zip=1
+ https://osmtools.de/josm/steps.xml
+ https://raw.githubusercontent.com/Flacus/Windrad/master/windrad.xml
+ https://josm.openstreetmap.de/josmfile?page=Presets/public_bookcase&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Surveillance&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/TMC&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/MobilePhoneBaseStations&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Golf_Course&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Animal_facilities&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/TurnLanes&zip=1
+ https://github.com/mrtngrsbch/Preset-GLAM-JOSM/archive/master.zip
+ https://josm.openstreetmap.de/josmfile?page=Presets/Historic_Stone&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Industrial&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Stolpersteine&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/StolpersteineLight&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Tagsforobservatories&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Windrad&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Maxspeed-zones&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Playground_Equipment&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/contact(socialnetworks_IMs)&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Disc_Golf_Course&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/MastAndTower&zip=1

Map paint styles:
- https://josm.openstreetmap.de/josmfile?page=Styles/PublicTransportV2&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/PTStops&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/PublicTransport&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Lane_and_Road_Attributes&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/AddressValidator&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Coloured_buildings&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Coloured_Streets&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Modified&zip=1
- https://raw.githubusercontent.com/species/josm-preset-traffic_sign_direction/master/direction.mapcss
- https://josm.openstreetmap.de/josmfile?page=Styles/DestinationSignRelation&zip=1
- https://raw.githubusercontent.com/yopaseopor/indoormap/master/indoormap-style.mapcss
- https://github.com/GlassOceanos/indoor-JOSM-style/archive/master.zip
- https://josm.openstreetmap.de/josmfile?page=Styles/Maxspeed&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/ParkingLanes&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Cycleways&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Schools&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Incline&zip=1
- https://raw.githubusercontent.com/yopaseopor/traffic_signs_style_JOSM/master/Styles_Traffic_signs_EUR.zip
- https://josm.openstreetmap.de/josmfile?page=Styles/Enhanced_Lane_and_Road_Attributes&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Landcover&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Admin_Boundaries&zip=1
- https://github.com/bastik/mapcss-tools/raw/osm/mapnik2mapcss/osm-results/mapnik.zip
- https://josm.openstreetmap.de/josmfile?page=Styles/NewHighwayColors&zip=1
- https://pasharm.github.io/New_basic_style_for_JOSM/New_basic_style.mapcss
- https://josm.openstreetmap.de/josmfile?page=Styles/NewParkingFeatures&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Surface&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/SimpleBuildingTags&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/SimpleRoofTags&zip=1
- https://raw.githubusercontent.com/yopaseopor/traffic_signs_style_JOSM/master/Styles_Traffic_signs_EUR_OC.zip

Validator rules:
+ https://josm.openstreetmap.de/josmfile?page=Rules/OsmoseValidations&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Rules/KeepRight&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Rules/GermanySpecific&zip=1

Last errors/warnings:
- 00000,452 W: extended font config - overriding 'filename.Mongolian_Baiti=monbaiti.ttf' with 'MONBAITI.TTF'
- 00018,228 W: java.io.IOException: Die angeforderte URL https://josm.openstreetmap.de/josmfile?page=Rules/OsmoseValidations&zip=1 wurde nicht gefunden
- 00018,340 W: Hinzufügen von https://josm.openstreetmap.de/josmfile?page=Rules/OsmoseValidations&zip=1 zum Merkmalsprüfer fehlgeschlagen
- 00018,341 W: java.io.IOException: No file found for: https://josm.openstreetmap.de/josmfile?page=Rules/OsmoseValidations&zip=1

[daganzdaanda]

The complete status report.

[simon04]

Replying to skyper:

Do we backup notes layers under deleted_layers?

Yes, see #11537

[simon04]

@daganzdaanda, running JOSM from the command line with the --debug flag will output a bunch of "opening new note", "adding comment to note X", ... lines, and print "Failed to upload note to server" when the upload fails.

However, when uploading a note fails, JOSM _should_ report "Notes failed to upload" at the end of the upload.

[skyper]

The situation might be more tricky when exiting JOSM. Did not check what happens when uploading (multiple) changesets or saving any files. and some action fails? JOSM should not exit but warn about the failed actions.

Replying to simon04:

Replying to skyper:

Do we backup notes layers under deleted_layers?

Yes, see #11537

Thanks, I figured this out already and it is documented.

[daganzdaanda]

@simon04 Thanks, I started JOSM from the command line with --debug added, but I only saw some static info on the main window background. Where should I look for the "opening new note", "adding comment to note X" lines?

[daganzdaanda]

... sorry, I was starting "javaw.exe" through the cmd, "java" alone works and gives feedback.

[daganzdaanda]

Hi again,

in the meantime, I found the "noteSolver" plugin and got mixed results from it.
I tried to narrow down an issue with it, and while doing this, I had some new cases where the changed notes were not uploaded even after telling JOSM so in the closing dialogue. This time I was able to document the problem a bit, see ​https://github.com/kmpoppe/noteSolver/issues/16#issuecomment-1090914132 and the following comments. I've got screenshots and included the --debug output and status reports.

I believe this might be the same thing that made me open this issue already when I did not have the noteSolver plugin installed. Maybe with the new information it's possible to get at the source of it now.

[daganzdaanda]

Some more ----maybe---- related info:

I just reproduced a bug where any note change is not uploaded with a single click on the "upload note changes to server" button, it only works the second time:

I closed JOSM and restarted to new session. Downloaded area and notes.
Click on "close note" button, add text.
Click on "upload note changes to server" button. Nothing happens.
Click a second time on that button: Note change is uploaded (commented and closed)

JOSM status: ​https://gist.github.com/daganzdaanda/751cb7e3d6c0c74d4b6047b1b9708a99
JOSM debug output: ​https://gist.github.com/daganzdaanda/3a8bfd860c95d4ff61093df4a901bc68
Hopefully there is some useful info in there.

See also the noteSolver issue mentioned above.
It made me think that something might not work right in initializing / finalizing the upload of note changes.
In this ticket, the call to upload came when JOSM was closing. In the noteSolver issue, the plugin sends the upload request. Now I tried with the "manual" upload button. The common thing is that the upload did not work after the first (and usually only) call. Maybe my JOSM is lazy and needs two "pushes" to go to work sometimes ;-)

[daganzdaanda]

Another test with a note around null island
[debug](​https://gist.github.com/daganzdaanda/70c41adab01d2c21972f968ad5cba0d8)
I needed 2 clicks again, but I can't interpret the debug output if it recorded the clicks or not.