Modify

Opened 9 months ago

Closed 9 months ago

Last modified 9 months ago

#18796 closed defect (fixed)

Validator stops finding all issues after upload

Reported by: skyper Owned by: GerdP
Priority: major Milestone: 20.03
Component: Core validator Version: latest
Keywords: template_report upload Cc:

Description

What steps will reproduce the problem?

  1. edit for a while with some validator runs and possibly uploads in between
  2. run validator without selection

What is the expected result?

Test are run on all objects according to user's settings.

What happens instead?

Not all test are run on all objects and a lot priory reported issues are missing.

Please provide any additional information below. Attach a screenshot if possible.

I mainly found the problem with FIXME=* not reported on info level but I fear there are more important issues missed.
I am working with incomplete data from overpass queries.
If I save the session and restart JOSM validator finds the issues again.

Relative:URL: ^/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2020-02-26 10:50:27 +0100 (Wed, 26 Feb 2020)
Revision:15937
Build-Date:2020-02-26 09:52:41
URL:https://josm.openstreetmap.de/svn/trunk

Identification: JOSM/1.5 (15937 en) Linux Debian GNU/Linux 10 (buster)
Memory Usage: 418 MB / 768 MB (12 MB allocated, but free)
Java version: 11.0.6+10-post-Debian-1deb10u1, Debian, OpenJDK 64-Bit Server VM

Java package: openjdk-11-jre:amd64-11.0.6+10-1~deb10u1
Java ATK Wrapper package: libatk-wrapper-java:all-0.33.3-22
libcommons-compress-java: libcommons-compress-java:all-1.18-2
libcommons-logging-java: libcommons-logging-java:all-1.2-2
fonts-noto: fonts-noto:all-20181227-1
liboauth-signpost-java: liboauth-signpost-java:all-1.2.1.2-2
VM arguments: [-Djosm.restart=true, -Djosm.dir.name=JOSM-latest, -Djava.net.useSystemProxies=true, -Dsun.java2d.opengl=True]
Dataset consistency test: No problems found

Plugins:
+ ImproveWay (29)
+ Mapillary (1.5.20)
+ OpenStreetCam (280)
+ OpeningHoursEditor (35250)
+ apache-commons (35092)
+ apache-http (35092)
+ comfort0 (35314)
+ conflation (0.6.4)
+ ejml (35122)
+ geotools (35169)
+ imagery_offset_db (35248)
+ jaxb (35092)
+ jna (35092)
+ jts (35122)
+ opendata (35330)
+ reverter (35313)
+ undelete (35313)
+ utilsplugin2 (35248)
+ waydownloader (35092)

Tagging presets:
+ https://josm.openstreetmap.de/josmfile?page=Presets/Crafts&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/LaneAttributes&zip=1
+ https://github.com/kendzi/Simple3dBuildingsPreset/releases/download/0.9_2018-05-08/s3db-preset.zip
+ https://josm.openstreetmap.de/josmfile?page=Presets/AdvertisingPreset&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Drinks&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/ParkingLanes&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Playground_Equipment&zip=1
+ ${HOME}/Downloads/FIXMEpreset.xml

Map paint styles:
+ https://josm.openstreetmap.de/josmfile?page=Styles/Lane_and_Road_Attributes&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Admin_Boundaries&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Styles/Bench&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Enhanced_Lane_and_Road_Attributes&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Styles/Incline&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Maxspeed&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Styles/Whitewater&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Styles/AdvertisingStyle&zip=1

Validator rules:
+ https://josm.openstreetmap.de/josmfile?page=Rules/GermanySpecific&zip=1

Attachments (1)

josm_18796_example.osm.bz2 (34.6 KB) - added by skyper 9 months ago.
example from last session

Download all attachments as: .zip

Change History (23)

comment:1 in reply to:  description Changed 9 months ago by Don-vip

Replying to skyper:

I am working with incomplete data from overpass queries.

Please attach a small sample file.

Changed 9 months ago by skyper

Attachment: josm_18796_example.osm.bz2 added

example from last session

comment:2 Changed 9 months ago by skyper

Sorry, the example is not small, though, it is the last file I noticed the problem.
The overpass query used quite a lot even on overlapping bboxes is:

[out:xml][timeout:90][bbox:{{bbox}}];
(
  way["waterway"];
  relation["waterway"];
  nwr["natural"="water"];
);
(._;>;);
out meta;

and yes, I did upload and then even use overpass afterwards on objects I might have uploaded short time ago.
After restart there is no problem with the file but I did not try to only save and load again without restart.

comment:3 Changed 9 months ago by skyper

I watched carefully this time and validator stopped to find/display warnings after upload. Saving the layer to file, deleting the layer and opening the file did not make any difference.

Last edited 9 months ago by skyper (previous) (diff)

comment:4 Changed 9 months ago by skyper

Keywords: upload added
Summary: Validator stops finding all issues after some time of editingValidator stops finding all issues after upload

comment:5 Changed 9 months ago by GerdP

Milestone: 20.03
Owner: changed from team to GerdP
Status: newassigned

comment:6 Changed 9 months ago by GerdP

Resolution: fixed
Status: assignedclosed

In 15975/josm:

fix #18796: Validator stops finding all issues after upload
Set/Reset the flag isBeforeUpload each time when test is executed

comment:7 Changed 9 months ago by GerdP

In 15976/josm:

see #18796: Validator stops finding all issues after upload
Set/Reset the flag isBeforeUpload each time when test is executed

comment:8 Changed 9 months ago by GerdP

In fact a flag was never reset and therefore some tests worked as in upload mode. This effected some tests in TagChecker and MapCSSTagChecker and the test UnconnectedWays when the preferences are set to skip those tests on upload.

Last edited 9 months ago by GerdP (previous) (diff)

comment:9 Changed 9 months ago by skyper

Keywords: regression added
Priority: normalmajor

comment:10 Changed 9 months ago by GerdP

Why is it a regression? I think it worked like this since r3669. Do you have different results?

comment:11 Changed 9 months ago by skyper

Just tested with r15268 and validator finds FIXME after upload. Something must have been changed in the last six months

Relative:URL: ^/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2019-08-01 21:17:11 +0200 (Thu, 01 Aug 2019)
Revision:15268
Build-Date:2019-08-02 01:31:02
URL:https://josm.openstreetmap.de/svn/trunk

comment:12 Changed 9 months ago by GerdP

Did you test this with the FIXME test disabled for upload?

comment:13 Changed 9 months ago by skyper

Yes, I used a clean preferences and only enabled "Info" for manual runs in preferences

comment:14 Changed 9 months ago by GerdP

All other checkboxes disabled? Or unchanged?

comment:15 Changed 9 months ago by skyper

Everything default except the "informal warnings" enabled. But wait a minute, made it a difference if validator did not find anything on upload?

Last edited 9 months ago by skyper (previous) (diff)

comment:16 Changed 9 months ago by GerdP

I can reproduce the problem wit r15268 like this:

  • start with clean preferences
  • enable info
  • disable FIXME checks on upload
  • create way with highway=secondary + fixme=position
  • run validator, make sure that message for fixme is shown
  • start upload, cancel when validator complains about missing name (or when asked for comment)
  • run validator again, note that message for fixme is not shown

comment:17 Changed 9 months ago by skyper

I can reproduce but:

Please, test with different settings:

  • start with clean preferences
  • enable info but not on upload and no other setting changes
  • create way with highway=secondary + fixme=position
  • run validator, make sure that message for fixme is shown
  • start upload, cancel when validator complains about missing name (or when asked for comment)
  • run validator again, note that message for fixme is still shown

So the problem got worse over time, though, it might not be a regression.

I just rechecked with a normal edit and for my settings I did not encounter this bug half a year ago.

Last edited 9 months ago by skyper (previous) (diff)

comment:18 Changed 9 months ago by GerdP

Yes, you are right. I'll check again if I really fixed the problem.

comment:19 Changed 9 months ago by GerdP

OK, r15682 made it worse.

comment:20 Changed 9 months ago by GerdP

So, more tests were no longer performed when the flag was incorrectly set.

comment:21 Changed 9 months ago by GerdP

I've corrected comment:8 accordingly. Before r15682 only MapCSSTagChecker and UnconnectedWays used this poorly initialised flag.

comment:22 Changed 9 months ago by skyper

Keywords: regression removed

No regression

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain GerdP.
as The resolution will be set.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.