Improve validation with data filters enabled

[ljdelight]

Users de-clutter their editing area by enabling filters to hide/disable objects, and these objects are sometimes not included when a user performs validation and are often hidden from view after validation actually occurs. This may skew the validation results without the user being aware of the issue.

This enhancement is to warn the user when validation is performed while filters are actively hiding/disabling objects. The dialog allows the user to persist their response as the saved preference key 'validator.vallidate_with_filters_action' (values are 'ask' (default), 'continue', and 'fail').

When the dialog should be shown to the user, here's a screenshot and the 'Help' button opens the JOSM Help Browser Validator page (​https://wiki.openstreetmap.org/wiki/JOSM/Validator ):

When the 'error' selection is saved, the next time the user runs validation with active filters this error message is shown:

The 'validator.vallidate_with_filters_action' Preference can be configured by changing the radio selection under Edit > Settings > Data validator:

[GerdP]

I was not aware that filters have an influence on the results of the validator. I think this should not happen, but maybe it's a wanted feature? I rarely work with filters. Can you give an example to reproduce this problem?

[Don-vip]

The error message should explain how to disable it in preferences.

[skyper]

What is that special about active filter with the indication in the top right corner of Mapview compared to validation with incomplete data, some selection or on upload?

[ljdelight]

Validation done with the filter enabled, causing the 'way end node near other highway' result to not be identified.

[ljdelight]

Replying to GerdP:

I was not aware that filters have an influence on the results of the validator. I think this should not happen, but maybe it's a wanted feature? I rarely work with filters. Can you give an example to reproduce this problem?

Sure, I attached 'ticket20729-filter-changes-validation-results.osm' with two ways that are near but not connected. The filter 'highway: residential' enabled and hidden does change the validation results from 3 warnings to 2 warnings, screenshots follow.
Additionally, I have seen cases where the validation creates objects (in the validation layer) that are not visible to the user and the dialog would help there.

Validation shows 3 warnings without a filter:

Validation shows 2 warnings with a filter (disable checked):

[GerdP]

OK, I see. The difference in the result is probably caused by the use of IPrimitive.isDrawable(). The implementations for this method return false for the filtered objects and thus some tests don't "see" all data. This is probably not intended, but I have to dig deeper to be sure. I'll open new tickets for those cases which seem wrong.

[simon04]

Only LongSegment and UnconnectedWays are affected. This isDrawable checks have been introduced in r5920 and r12032, respectively.

[GerdP]

The code in Multipolygon also checks isDrawable and some tests use this code.

[GerdP]

This has some more ugly effects. If you have a filter activated that shows only natural=tree and load a file containing some multipolygons and then deactivate the filter the multipolygons are not rendered properly.

[GerdP]

I think Multipolygon.load() should not checkisDrawable(). I assume isUsable() was meant here and also in the other tests.

[GerdP]

My proposal to fix the tests by replacing sDrawable() by more appropriate methods

[simon04]

+1. Sounds reasonable. Time will tell if isDrawable was there for a good reason...

[GerdP]

In 17761/josm:

see #20729:Alert user if validation action is performed with data filters enabled

• replace isDrawable() by more appropriate methods to make results of validator independent of filter settings

TODO: use MultipolygonCache.get() instead of new Multipolygon() in more places to improve performance.

[GerdP]

@ljdelight: I think r17761 solves part of the problem.
Do you see other possible reasons why filters cause different test results? I looked for isDisabled and other methods which depend on the filter but found no more issues in core.

[ljdelight]

Hey @GerdP, your patch solved what I had noticed with the different test results.
The remaining issue is the case where the validation creates objects that are not visible to the user (comment 6 has the screenshots).
What are your thoughts on the dialog?

[GerdP]

I see no reason to show a warning before the validation is executed now that the errors are fixed. Maybe just show a hint when the validation found problems with hidden objects? Or show a hint when such a problem is selected?

[ljdelight]

Okay I hacked together a simple dialog that displays after validation is done and a TestError includes primitives that aren't drawable.

I'm not familiar with a hint, do you have an example? Thank you

[GerdP]

Look for Notification

[ljdelight]

Example of the notification

[ljdelight]

That's slick, here's what it can look like:

Current patch at this point would be josm-show-notification-on-hidden-objects.patch​

[GerdP]

I think the text is too complex and missleading as the validation layer doesn't show markers for filtered objects.
What about Validation results contain elements hidden by a filter. for the first line?

This also works for users that deactivate the "Validation errors" layer. Still, they may note problems when the "Zoom to problem" action shows an empty area.

[GerdP]

sligtly simplified patch with different text

[ljdelight]

thanks @GerdP

[simon04]

Considering the milestone:21.04 roadmap, we probably should no longer change i18n strings...

[GerdP]

Yes, I also want to wait.

[ljdelight]

Hi, is there anything else needed on this ticket? It's in a need info. Thanks

[GerdP]

No, I'll commit my patch after release of 21.04 unless anybody suggests a better text.

[skyper]

What do you want to say with "review"? Is "disable" better?

Please disable active filters to see the hidden results.

[GerdP]

My understanding is that the filters may stay activated, only if the checkbox below H is enabled nothing is rendered and "zoom to problem" might show an empty area.

[GerdP]

In 17852/josm:

fix #20729: Improve validation with data filters enabled

• show notification when validator was executed with filters and at least one element in the error message is hidden