In addition to "conflicts", detect and resolve "potential map anomalies"

[skyper]

Hi
Thanks for your work !!!!

I do not know which of the previous reported bugs apply for this bug, or are their even more ?
Fact is that merging results in broken data. I have attached 3 osm-files. First my one, second data just downloaded, third the result of merge after conflict-resolution (only 10 conflicts, I think their should be more !!!). Check for id:48279045 and this intersection.

By the way, lots of the duplicated nodes in the server.osm are result of josm not uploading right. (#4401)

cu skyper

[skyper]

3 osm

[mjulius]

What is technically wrong with way 48279045?

Besides that the whole intersection is somewhat ugly and confusing.

[skyper]

Replying to mjulius:

Besides that the whole intersection is somewhat ugly and confusing.

Please have I look at "my" file. I worked over this intersection, trying to get it clean, but merging always destroys my data !![[BR]] But yeah this is confusing. From the trunk you have two lanes going left. The right one goes also straight and for the left one U-Turn is possible.

[mjulius]

Replying to skyper:

Replying to mjulius:

Besides that the whole intersection is somewhat ugly and confusing.

Please have I look at "my" file. I worked over this intersection, trying to get it clean, but merging always destroys my data !!!

Does merging really destroy your data? If objects have a new version on the server and you have modified your local version JOSM should create a conflict. If JOSM does not have local modifications to the object it is replaced by the version from the server. And if a node that you have used in a new or modified way has been moved by someone since you got your version of it your way will change its shape. What else do you expect to happen?

This is exactly the same result you get when you upload your new way to the server. Because you did not change the node the server will not detect a conflict. You only get to see the result after the renderers have updated their maps.

And this is why it is a good idea to update your data frequently and before uploading - and not only the data with conflicts. Depending on the activity in your area.

JOSM could be modified to add a conflict when a node has changed location that is being used in a new or modified way. I am not sure it would be a good idea to do because you might get a lot of conflicts if someone realigned a whole way to his new GPS track.

[jttt]

I think there should be conflict when modified way changed it's shape. The way should be in conflict, not the moved nodes. I would even say that there should be no conflict for node, if it's part of only one way and has no tags.

Unfortunatelly that would require quite big refactoring of conflict handling.

[skyper]

+1

I do not update my data using "update data" because in the third step of update I can only choose to have the deleted objects on the server deleted on my data, too, or ignore it totally. I have no possibility to decide by myself (conflicts). I would like to have an option to have all these conflict shown and not silently solved. That is why I always download the area manually an then merge.

Someone might have accidently deleted a way you draw a month ago. If you "update data" this way is deleted and unlike potlatch you do not have any possibility to revert this change and even do not get much information about the silently deleted objects.

I do not care about anything changes in the outside downloaded area and I do not mind to have nodes without tag/connection been deleted silenty, but for all other changes please provide a possibility to either keep your data or at least open conflicts.

Thanks

[stoecker]

The conflict handling is not designed to revert changes. It is meant to allow fixing conflicting changes done during your editing time. I object against changing the target of conflict handling. The way it works is the commonly accepted method. Maybe details can change, but not the general idea.

What you request is an entirely different topic. We are working on ways to get previous data states back, but essentially some server support is still missing to get this done.

This is a collaborative project, so changes are done by multiple users and not your local files, but the server state counts. So JOSM gives the server higher value when data does not match and is not modified locally. This is absolutely correct.

[skyper]

I get your point, but I still think there should be a possibility to manually solve these conflicts besides having josm auto-solve. Right now it is quite frustating to upload with josm-latest working offline with a slow machine works nice so. I do not have much time online, so I do many changes offline. I used to just "update data" and let josm delete staled objects, until I discovered that many of my previous changes where deleted (housenumbers for example). I was lucky and had some backups. (This way I got the IDs and was able to contact the responsable users.)

Right now I have around 10 osms waiting to upload but since conflict-management and merging osm is/was not well working. I tried to keep up with the server downloading the area again and the merging it with my changes offline to upload it the next day, but this did not work well and took me also long time.

I know the that there are quite a lot squarekilometers on our lovely planet where the is and will be none or really slow internet and even if it exists, it is only for some hours a day.

Thanks a lot for your work. (Sorry, I am not that into java and have not that much time to help.)
Regards skyper

[Gubaer]

Replying to jttt:

I think there should be conflict when modified way changed it's shape. The way should be in conflict, not the moved nodes. I would even say that there should be no conflict for node, if it's part of only one way and has no tags. Unfortunatelly that would require quite big refactoring of conflict handling.

Yes, because the criteria for whether we have to deal with a conflict or not is currently purely "local". The decision is entirely based on what we know from the source and the target dataset (the dataset we merge from and the dataset we merge to). If we introduce a condition "if it's part of only one way" we would probably have to query the parents of every node participating in a way.

[Gubaer]

I do not update my data using "update data" because in the third step of update I can only choose to have the deleted objects on the server deleted on my data, too, or ignore it totally. I have no possibility to decide by myself (conflicts). I would like to have an option to have all these conflict shown and not silently solved. That is why I always download the area manually an then merge.

This looks like a write up for an enhancement, I've moved it to a new ticket, see #4518.

[Gubaer]

I'd like to rephrase mjulius' basic questions: what's wrong with the data? In what sense is it "broken"?

In a technical sense, in my opinion, it isn't: * I can load Merge_Bug_my.osm into JOSM * way 48279045 looks just fine, both on MapView as well as in the OSM file.

This is classified as blocker and I'd like to have a better understanding what is actually wrong, because most of the comments in this thread are about how conflict detection and conflict resolution could be improved to better support offline editing.

[jttt]

Replying to Gubaer:

Replying to jttt:

I think there should be conflict when modified way changed it's shape. The way should be in conflict, not the moved nodes. I would even say that there should be no conflict for node, if it's part of only one way and has no tags. Unfortunatelly that would require quite big refactoring of conflict handling.

Yes, because the criteria for whether we have to deal with a conflict or not is currently purely "local". The decision is entirely based on what we know from the source and the target dataset

That's enough for nodes. If you download the edited area from server than all nodes and ways will be available locally.

Replying to Gubaer:

I'd like to rephrase mjulius' basic questions: what's wrong with the data? In what sense is it "broken"?

Wrong is that edited way changed it shape to complete nonsense and there was no conflict for it. In this case skyper spot it because there was conflict for one of the way nodes. But it's possible that there will be no conflict at all.

For example when user adds existing node to the way and the node change position on the server. Then there will be way with changed shape and user will not be notified about it.

[anonymous]

Replying to Gubaer:

Replying to jttt:

I think there should be conflict when modified way changed it's shape. The way should be in conflict, not the moved nodes. I would even say that there should be no conflict for node, if it's part of only one way and has no tags. Unfortunatelly that would require quite big refactoring of conflict handling.

Yes, because the criteria for whether we have to deal with a conflict or not is currently purely "local". The decision is entirely based on what we know from the source and the target dataset

That's enough for nodes. If you download the edited area from server than all nodes and ways will be available locally.

Replying to Gubaer:

I'd like to rephrase mjulius' basic questions: what's wrong with the data? In what sense is it "broken"?

Wrong is that edited way changed it shape to complete nonsense and there was no conflict for it. In this case skyper spot it because there was conflict for one of the way nodes. But it's possible that there will be no conflict at all.

For example when user adds existing node to the way and the node change position on the server. Then there will be way with changed shape and user will not be notified about it.

[Gubaer]

For example when user adds existing node to the way and the node change position on the server. > Then there will be way with changed shape and user will not be notified about it.

I think this is actually a wish for detecting something for which I coin the term "potential map anomaly". What's the difference with a "conflict" as we detect and resolve it in JOSM now?

• a conflict is a purely techical collision. On the level of an individual object, JOSM isn't able to decided whether a source or a target version of the same object has precedence. It needs user intervention. Therefore it generates a conflict.

• a potential map anomaly is some kind of "semantic disorder" in the map data. It's a collision of the intentions of two mappers, the intention of the mapper responsible for the source version of the object(s) isn't compatible with the intention of the mapper of the target object(s).

• Example: after an update one of the four corners of a nice rectangular building is moved 10m north and the nice shape of the building is badly distroyed. JOSM won't show a warning if this happens (unless it detects a "conflict", of course, but assume for the sake of the example that it doesn't.

JOSM is good in detecting and resolving conflicts and it requires users to resolve them because otherwise the data is invalid.

JOSM is bad in detecting "potential map anomalies" (actually, it doesn't support it at all). Even if was better at it, it wouldn't require users to resolve them. Even if a merge operation led to "potential map anomalies" the data would be perfectly valid.

After all, I think that this ticket and the discussion leads to an interesting new feature in JOSM. In addition to "conflicts" JOSM should support "potential map anomalies" in the future:

• it should detect them during a merge. We would need good heuristics. For instance, not every moved node in a way should trigger a "potential map anomalie", just the "relevant" ones.
• it should provide a "map anomalie dialog" with a list of anomalies. Mappers can consult this list, check the anomalies, fix them or dismiss them. The challenge will be to support this with a good UIs, mainly a UI which allows to visually compare the state before and after a merge.

Regarding this ticket: I'm turning it into a major enhancement. It isn't blocking defect in the sense that "normal" work with JOSM isn't possible if it doesn't get fixed. It addresses a missing functionality in JOSM which makes working with JOSM harder in an advanced mode.

[skyper]

Sorry for making so much work, but weekend is over and I am back again.

I think there is still a bug in merging. I have to older osm of the same region. I think the only thing I edit is deleting errors (overlaping way, doubled nodes). I do not find any id:0.

I merge this files and only get conflicts of nodes, but there exist ways in one osm that do not exist in the other one. I think there should be an conflict on but please tell me if I am wrong. Look for the round-about "Siegesdenkmal" and there at the low left. I also noticed that there is no conflict of a relation, but this way (42667459) still exists but lost its membership.

I merged the older file onto the new, but I think it also is not working right the other way round. May we have to open another ticket.

Cheers skyper

[skyper]

2 old osm

[Gubaer]

I've created a new ticket for the last comment from skyper, see #4543.

Removing r-2010-01-blocker again and changing into an enhancement, see above.