Provide merge support for ways and relations

[anonymous]

I have done a laod of mapping from this weekends Milton Keynes mapping party, and after hours and hours of tracing, I decided to upload my hard work. Here is the error message I got:

Version mismatch: Provided 7, server had 8: of way 4016377

What do i have to do to solve this?

[parker.cj@…]

Version Mismatch problem

[Gubaer]

• download from the server before you upload. JOSM will try to merge the the newer version on the server (version 8) with your version (version 7)
• JOSM will probably create a conflict because the list of nodes in version 8 and in version 7 are different. Unfortunatelly, JOSM can't help you yet to resolve it, though.

You can try to resolve the conflict using a text or XML editor.

• save your local dataset locally, say in mydata.osm
• open mydata.osm in a text editor

• search for <way id="4016377

• the output of ​http://www.openstreetmap.org/api/0.6/way/4016377 is the server version of the way as XML

This could be your version

<way id="4016377" visible="true" timestamp="2009-05-18T09:26:44Z" 
user="EdLoach" uid="57884" version="7" changeset="1233860">
<nd ref="10317666"/>
<nd ref="10317951"/>
<nd ref="10317937"/>
<nd ref="10317943"/>
<nd ref="10317632"/>
<nd ref="17673076"/>
<nd ref="10317631"/>
<nd ref="10317936"/>
<nd ref="10317941"/>
<nd ref="10317945"/>
<nd ref="10317935"/>
<nd ref="10317939"/>
<nd ref="10317634"/>
<nd ref="17673081"/>
<nd ref="10317633"/>
<nd ref="10317956"/>
<nd ref="10317938"/>
<nd ref="10317954"/>
<nd ref="10317666"/>
<tag k="created_by" v="Potlatch 0.10f"/>
<tag k="highway" v="tertiary"/>
<tag k="junction" v="roundabout"/>
<tag k="name" v="South Seclow Roundabout"/>
</way>

This is the server version

<way id="4016377" visible="true" timestamp="2009-05-18T20:54:01Z" 
version="8" changeset="1242297" user="Andrew Chadwick" uid="27408">
<nd ref="359049030"/>
<nd ref="10317951"/>
<nd ref="10317937"/>
<nd ref="10317943"/>
<nd ref="10317632"/>
<nd ref="17673076"/>
<nd ref="10317631"/>
<nd ref="10317936"/>
<nd ref="10317941"/>
<nd ref="10317945"/>
<nd ref="10317935"/>
<nd ref="10317939"/>
<nd ref="10317634"/>
<nd ref="17673081"/>
<nd ref="10317633"/>
<nd ref="10317956"/>
<nd ref="10317938"/>
<nd ref="10317954"/>
<nd ref="359049030"/>
<tag k="name" v="South Seclow Roundabout"/>
<tag k="created_by" v="Potlatch 0.10f"/>
<tag k="highway" v="tertiary"/>
<tag k="junction" v="roundabout"/>
</way>

• manually merge the list of <nd > elements in the way. This could result in

<way id="4016377" visible="true" timestamp="2009-05-18T20:54:01Z" 
version="8" changeset="1242297" user="Andrew Chadwick" uid="27408">
<nd ref="359049030"/>
<nd ref="10317951"/>
<nd ref="10317937"/>
<nd ref="10317943"/>
<nd ref="10317632"/>
<nd ref="17673076"/>
<nd ref="10317631"/>
<nd ref="10317936"/>
<nd ref="10317941"/>
<nd ref="10317945"/>
<nd ref="10317935"/>
<nd ref="10317939"/>
<nd ref="10317634"/>
<nd ref="17673081"/>
<nd ref="10317633"/>
<nd ref="10317956"/>
<nd ref="10317938"/>
<nd ref="10317954"/>
<tag k="name" v="South Seclow Roundabout"/>
<tag k="created_by" v="Potlatch 0.10f"/>
<tag k="highway" v="tertiary"/>
<tag k="junction" v="roundabout"/>
</way>

• replace the definition of way 4016377 in mydata.osm with the merged fragment and save. make sure the version of the way in mydata.osm is 8 !
• load in OSM and upload

[anonymous]

Thanks.

The good news
You fixed my problem

The bad news
There is another problem that is the same. How did you find the server data? Is there a way I can do it so in the future I can just fix this?

Just wondering, is this caused by my bad editing, a problem with JOSM or a problem with the server.

Thanks

[Gubaer]

When you get a Version mismatch: Provided 7, server had 8: of way 4016377 you can inspect the version on the server either through the browse API or the OSM API:

• ​http://www.openstreetmap.org/browse/way/4016377 displays details about the way in the browser, with ​http://www.openstreetmap.org/browse/way/4016377/history you get information about its history. Replace way with node or relation if the OSM primitive with the version mismatch is a node or a relation.

• ​http://www.openstreetmap.org/api/0.6/way/4016377 replies the same information as XML. This is the data JOSM loads and compares with the local dataset. Use ​http://www.openstreetmap.org/api/0.6/way/4016377/7 to get version 7. Replace way with node or relation if the OSM primitive with the version mismatch is a node or a relation.

Just wondering, is this caused by my bad editing, a problem with JOSM or a problem with the server.

Don't worry, it's not your fault. The server is fine, too. It properly checks whether your edits could interfere with the edits of another OSM mapper (it's called "optimistic locking"). I'd say it's a problem with JOSM, it doesn't help users to resolve this kind of conflicts, yet. In JOSM it's not exactly a defect, rather a missing feature nobody felt responsible to code so far.

[Gubaer]

Provide merge support (in terms of a merge dialog) for ways and relations. In particular,

• merge support for different node lists in different versions of a way
• merge support for different member lists in different versions of a relation

[parker.cj@…]

Ok, getting there slowly. I have just spent the last few hours resolving the version mismatches. I think I have got them all. At least, there are no more coming up. However, im not out of the woods yet. I now get this error message:

"Precondition failed: Node 359051303 is still used by way 34731225."

How do I fix this? Im not sure what to look for and fix (the way or the node?)

[stoecker]

That's another problem. JOSM currently cannot detect elements which have been deleted on the server. Node 359051303 has been deleted on the server, but you local variant still includes it. Solution is to remove the references to this node in you OSM file also.

It really gets time to find a solution for these conflicting changes problem. Volunteers?

[Gubaer]

I'm aware of it since API 0.6 switch and I'm working on it. Patches to be available really soon ;-)

[stoecker]

Ticket #2521 has been marked as a duplicate of this ticket.

[stoecker]

Ticket #2512 has been marked as a duplicate of this ticket.

[stoecker]

Ticket #2529 has been marked as a duplicate of this ticket.

[stoecker]

Ticket #2528 has been marked as a duplicate of this ticket.

[stoecker]

Ticket #2616 has been marked as a duplicate of this ticket.

[stoecker]

Ticket #2523 has been marked as a duplicate of this ticket.

[stoecker]

Set a set of other tickets as duplicate. They all are different aspects of the same problem: Data has changed during editing. Specific following issues must be handled:

• elements have been deleted on server (how to detect this and update JOSM is currently unclear)
• concurrent modifications.

[stoecker]

Ticket #2449 has been marked as a duplicate of this ticket.

[stoecker]

Ticket #2468 has been marked as a duplicate of this ticket.

[smsm1]

Replying to stoecker:

Set a set of other tickets as duplicate. They all are different aspects of the same problem: Data has changed during editing. Specific following issues must be handled:

• elements have been deleted on server (how to detect this and update JOSM is currently unclear)

See this thread on the josm dev list about the solution:
​http://lists.openstreetmap.org/pipermail/josm-dev/2009-April/002936.html
Basically mark all elements dirty before the map call. For any items in that are are downloaded in the map call mark them as clean. Then for all the items that are still dirty you need to do a multi get.

[stoecker]

Actually that solution is not so fine. What would be best is an API call to get all deleted elements of a certain area. This would also be required for history support to undelete stuff.

[Gubaer]

Replying to stoecker:

Actually that solution is not so fine. What would be best is an API call to get all deleted elements of a certain area. This would also be required for history support to undelete stuff.

Of course, a dedicated API call would be best. But there is no such call in API 0.6 and the multi GET approach is what was more or less accepted on the dev list. We have to avoid individual GETs and hitting the server to often. Otherwise, there are people on the server team who would be willing to ban JOSM

[smsm1]

Replying to stoecker:

Actually that solution is not so fine. What would be best is an API call to get all deleted elements of a certain area. This would also be required for history support to undelete stuff.

What do you not like about the solution on the mailing list?

[stoecker]

Well, banning the most active editor would surely help OSM to go forward. I had a look at the Changesets recently and wheras Potlatch creates most of them (and at least half of them without a single change) the changes done with JOSM still outweight all other editors largely :-)

to 21:
It is a workaround and we anyway need the deleted elements. I ask myself why 0.6 did not implement the necessary call. As Potlatch has access to these it should be obvious others want that also.

When we do that "mark bad" and update in the download refresh process I can live with it as long it is done for the current download window and not the whole data set.

Nevertheless we need an "refresh only the modified/touched fields before upload". Updating the whole dataset is a crude workaround.

[Gubaer]

As Potlatch has access to these it should be obvious others want that also.

Do you mean there is an unofficial API call for deleted elements and that Potlatch is already using it?

[smsm1]

Replying to Gubaer:

As Potlatch has access to these it should be obvious others want that also.

Do you mean there is an unofficial API call for deleted elements and that Potlatch is already using it?

AMF controller has it.

[Gubaer]

AMF controller has it.

I'm lost. "AMF controller" ?

[smsm1]

Replying to Gubaer:

AMF controller has it.

I'm lost. "AMF controller" ?

Potlatch's special API, which will hopefully disappear with the next major version of Potlatch.

[Matt]

Replying to stoecker:

It is a workaround and we anyway need the deleted elements. I ask myself why 0.6 did not implement the necessary call.

because nobody asked for it. certainly, in the discussions we've been having over the past year, i can't remember anyone saying it was necessary.

As Potlatch has access to these it should be obvious others want that also.

i'm sorry - it wasn't obvious to me. apparently it was so obvious to everyone else that nobody bothered to mention it!

as shaun says, we'll be trying hard to make sure that "the API" is the only API. the AMF stuff just makes it harder for us to develop, test and document the code.

can i ask what kind of API call you would like to see? is it as simple as adding a "?deleted=true" argument to the map call, or would you like something more complex?

[stoecker]

I think to solve most problems it would be an "give me last version of all deleted objects inside boundary box xxx". Probably this should also include the option to have "deleted since yyy". Probably cases where someone wants access to elements deleted several month ago are very seldom. Probably this can also be combined with normal download to save API calls.

For history handling (revert deletions) that should help a lot (as from this base everything else can be done) and it would also help to fix the problem this bug is about in a nice way.

Nevertheless for history purposes a "give me the state from xxx" would be necessary, but this is much more complicated. I think an API call to get information about deletions will give enough work for next time and helps to learn how a better support for "data reverts" may look like.

[smsm1]

You would still need to do a multiget for anything that has been moved out of the current bbox.

[Matt]

Replying to stoecker:

I think to solve most problems it would be an "give me last version of all deleted objects inside boundary box xxx".

we can easily do this as part of the map call - i will bring it up on the dev list.

Probably this should also include the option to have "deleted since yyy". Probably cases where someone wants access to elements deleted several month ago are very seldom. Probably this can also be combined with normal download to save API calls.

this is much harder, as it would involve changing the queries a lot more.

For history handling (revert deletions) that should help a lot (as from this base everything else can be done) and it would also help to fix the problem this bug is about in a nice way.

i agree, although multiget should also be done on all modified objects. as shaun pointed out, the map + deleted-map calls won't tell you when something has moved out of the bbox.

Nevertheless for history purposes a "give me the state from xxx" would be necessary, but this is much more complicated. I think an API call to get information about deletions will give enough work for next time and helps to learn how a better support for "data reverts" may look like.

for map calls this is unlikely to happen, but you can get the state at a particular time from the history call on the element you're interested in.

[Gubaer]

If you are going to work on the API would it be possible to extend PUT /api/0.6/changeset/#id too?

Currently,

• if a changset includes an update of a deleted node the server replies a 409 Conflict
• if a changeset includes a delete of a deleted node the server replies a 410 Gone (BTW: this is not yet documented as error response code for PUT /api/0.6/changeset/#id)

It would be nice if we got a 410 Gone in both cases. Otherwise we would have to query the respective node with a GET in order to know whether we just run into conflicting tags/nodes/relation members or whether the node was deleted.

[Gubaer]

Ticket #2637 has been marked as a duplicate of this ticket.

[Gubaer]

Ticket #2628 has been marked as a duplicate of this ticket.

[Gubaer]

Ticket #2627 has been marked as a duplicate of this ticket.

[Gubaer]

There is an experimental feature for merging ways in r1622.

• add conflict.resolution=extended to your prefs
• the traditional conflict resolution dialog will be replaced by an extended dialog which allows for merging conflicting node lists of ways

[Gubaer]

Now includes merge support for relations, too. See r1631.

[Gubaer]

See r1642 for an update:

• added row numbers to the dialog for conflict resolution in node lists and member lists
• added synchronized scrolling in the tables for my, their, and merged entries
• fixed bugs

[Gubaer]

See r1650 for an update:

• added concept of "merge pairs" (my vs. merged, my vs. their, their vs. merged)
• added highlighting of conflicts with colors (green=same entry at same position in opposite list, orange=same entry at different position in opposite list, red=entry not included in opposite list)
• now displays number of entries for lists
• fixed bugs