JOSM should store and use the information that parents were downloaded

[skyper]

Follow up from #18528, see GerdP in comment 19.

Best would be to store this information in a new flag.

• Currently, JOSM does not store information that parent objects where downloaded which can lead to additional traffic.
• There a many situations where the download area is used to determine if all parents of an object have been downloaded which is not the best solution in many situations.

I know this is quite some work and might need some time. Thanks in advance.

Use case / Example

• download parents
• all split and combine way actions
• moving objects, see #10999 for one example.
• validator test with parents involved
• to be continued

[GerdP]

My thoughts:

• Let's assume we implement a new flag PARENTS_KNOWN and new getter/setter methods like boolean parentsKnown() / void setParentsKnwon(boolean flag)
• Let's assume that the flag is set to true for all objects which are downloaded from a server that makes sure that parents are complete:
  1. Retrieving map data by bounding box: GET /api/0.6/map or the corresponding overpass query
  2. the Download Parents action would set the flag for the object for which the parents were retrieved
• This would allow to improve all actions which change objects in a way that parents might be corrupted, esp. split / join actions. Those actions would ask for confirmation that missing informaion should be downloaded if the flag is not set.
• When we load a file from disk (either in OSM format or something else which is converted to OSM) and this file contains one or more bounding boxes (bounds) we have to decide if the data within those bounds is considered complete or not. Several tools like osmfilter or even a simple text editor allow to change or remove data in the file. We use this to producde test data. So, to stay on the safe side we might decide to distrust any data loaded from disk, means, the new flag would not be set. Obviously this could cause confusion when mappers save their work to disk and reload it. Validator results might change.
• We would need a new file format that allows to store the flag. Without such a possibility we will not be able to reproduce certain situations when debugging or testing. The format should also contain a checksum.

[skyper]

Do not see any differences between saving and trusting bboxes, atm, and saving and trusting the flag. You will always be able to modify the file outside of JOSM.

More interesting is the update mechanism. How should it work for update data/modified/selection. Right now, these are the actions to use with saved (outdated) data.

An edge case and own story which needs to be check on current release, too, is the situation if a way is split while one the server a new relation with the way as member is created. Do not think there is a check and a conflict created, atm.

[GerdP]

Replying to skyper:

Do not see any differences between saving and trusting bboxes, atm, and saving and trusting the flag. You will always be able to modify the file outside of JOSM.

Assume you move a node which has the flag set outside the bbox and save to file. On reload the flag is not set. Can lead to different results in validator tests or when debugging a problem that is related to the flag.

More interesting is the update mechanism. How should it work for update data/modified/selection. Right now, these are the actions to use with saved (outdated) data.

When you update a selection the flag cannot be calculated, so it would either be set to false (parents unknown).
If you update a bbox the flag could be set.

An edge case and own story which needs to be check on current release, too, is the situation if a way is split while one the server a new relation with the way as member is created. Do not think there is a check and a conflict created, atm.

There is no conflict but the relation might be broken afterwards. No way to avoid that with the optimistic locking used by OSM.

[skyper]

Replying to GerdP:

Replying to skyper:

Do not see any differences between saving and trusting bboxes, atm, and saving and trusting the flag. You will always be able to modify the file outside of JOSM.

Assume you move a node which has the flag set outside the bbox and save to file. On reload the flag is not set. Can lead to different results in validator tests or when debugging a problem that is related to the flag.

I wrote saving the flag and I meant the differences between trusting bbox in file (now) and trusting known parents flag in file (future?)

An edge case and own story which needs to be check on current release, too, is the situation if a way is split while one the server a new relation with the way as member is created. Do not think there is a check and a conflict created, atm.

There is no conflict but the relation might be broken afterwards. No way to avoid that with the optimistic locking used by OSM.

Relation will be broken but an enhancement for api v0.7.

[stoecker]

That whole idea does not bring any advantage. There are simply too many cases which still can fail:

• data modified outside JOSM
• files created outside JOSM
• changes on the server after JOSM downloads
• special download situations in case of partial downloads

I don't see any sense in following this path any further. Such a download flag does not solve a single problem in a reasonable way, but only makes the code more complex, wastes memory, requires format changes and so on.

A lot of complexity for nothing.

Everybody not downloading data from the main OSM API should be aware of the effects this can have. If that is not clear enough then make e.g. notification inside the Overpass API more explicit.

Useful ways to follow could be:

• More expliciti user information in JOSM
• request for better download feature for "parent" updates in the API.

Adding a special case flag is not the right way.

[skyper]

See mdk`s comment 11 and followups for similar thoughts.

[skyper]

So the solution is to always check the downloaded area and offer to download parents first if outside of it (or if no download area exists)?

[GerdP]

I think: We should not allow several modifications:

• an old node which is not inside a download area should not be changed, at least it should not be deleted or moved. Maybe adding a tag is no problem, removing tags is dangerous (mapper might think that railway=crossing is wrong when they don't see the rails.
• it should not be possible to add a node to an old way when this new node is not inside a download area.
• it should not be possible to unglue an old node which is not inside a download area
• it should not be possible to split an old way at a node which is not inside a download area

(The list might not be complete)
I think it is the wrong approach to add lots of code in every action which might do any of the above actions to show dialogs and ask users if they really want to do something potentially wrong and even more code to show warnings or conflict dialogs when the users say yes.

Maybe we can add code to the UndoRedoHandler instead which could check if any "forbidden" command was produced and then show a notification that this modification requires a download area?

[skyper]

Working on route relations, I usually only use "download parents" and for nodes parents even only a new layer which I delete directly after getting the clue what/where I am editing.

I kindly ask to not completely forbid to edit outside download area, at least in expert mode. I already do not like the strictness of "combine ways" and usually work around it by deleting the ways but one and then extending the remaining way.

Do not forget, that there are situations where editing new object with id:0 is also dangerous, like splitting a way and then working with the "new way".

I think, notifications instead of warnings is the way to go, as warnings will probably bother too much and therefore will be ignored.
Anyway I fear, I will ignore the notifications, too, once I get it for every action outside of download area, especially, after downloading parents in advance. So the real problem will remain that there is no information if all parents have been downloaded.

[GerdP]

The real problem is that you want idiot proof support for an error prone workflow ;)