#12706 closed defect (fixed)
Sync remaining maps from EII and JOSM
Reported by: | stoecker | Owned by: | team |
---|---|---|---|
Priority: | normal | Milestone: | |
Component: | External imagery source | Version: | |
Keywords: | imagery maps EII | Cc: | bastiK |
Description
The image list at ImageryCompare contains still a number of red entries. They should either be added to JOSM's wiki or marked to be ignored.
If done we can automate add results into the automated build checking process.
See also these tickets:
Attachments (0)
Change History (57)
comment:1 by , 9 years ago
Milestone: | 16.04 → 16.03 |
---|
comment:2 by , 9 years ago
Milestone: | 16.03 → 16.04 |
---|
comment:3 by , 8 years ago
Milestone: | 16.04 → 16.05 |
---|
comment:4 by , 8 years ago
Milestone: | 16.05 → 16.06 |
---|
comment:5 by , 8 years ago
Milestone: | 16.06 |
---|
comment:6 by , 8 years ago
What do the = 1
and = 3
behind the ignore entries mean? https://josm.openstreetmap.de/browser/josm/trunk/scripts/sync_editor_imagery_index.groovy?rev=9653#L97
comment:7 by , 8 years ago
And what is about the entries at the end of ImageryCompare?
+++ Obsolete skip entry:
I don't see a common schema there.
comment:8 by , 8 years ago
EII uses: Hike & Bike - http://toolserver.org/tiles/hikebike/{zoom}/{x}/{y}.png
JOSM uses: Hike & Bike - http://{switch:a,b,c}.tiles.wmflabs.org/hikebike/{zoom}/{x}/{y}.png
Both work. What's the difference? Which is "better"?
follow-up: 10 comment:9 by , 8 years ago
What do the = 1 and = 3 behind the ignore entries mean? https://josm.openstreetmap.de/browser/josm/trunk/scripts/sync_editor_imagery_index.groovy?rev=9653#L97
The number of lines to skip.
And what is about the entries at the end of ImageryCompare?
+++ Obsolete skip entry:
I don't see a common schema there.
There are entries to ignore differences in the script, but nothing was ignored. That's an indication that the entry got changed in between. E.g. an Country code was added or something alike or in best case the difference has been resolved.
Either correct the text of the ignore (when a similar unignored entry exists which is not better than JOSM's entry) or simply remove it (when it has been resolved).
EII uses: Hike & Bike - http://toolserver.org/tiles/hikebike/{zoom}/{x}/{y}.png
JOSM uses: Hike & Bike - http://{switch:a,b,c}.tiles.wmflabs.org/hikebike/{zoom}/{x}/{y}.png
Both work. What's the difference? Which is "better"?
Toolserver is an very old address (go to Hike&Bike map and see what it uses), maybe also using old data. That's why it's better to fix it on both sides instead of using the ignores, so the question needs to be asked only once: But I personally did't want to make EII submit requests for data - in this point I'm petulant - I'll work with them, but not for them.
comment:10 by , 8 years ago
Replying to stoecker:
What do the = 1 and = 3 behind the ignore entries mean? https://josm.openstreetmap.de/browser/josm/trunk/scripts/sync_editor_imagery_index.groovy?rev=9653#L97
The number of lines to skip.
Which lines?
comment:11 by , 8 years ago
Could we please have one more category "updates which need to be solved in EII lists" so we can better distinguish all the entries?
Does the current ignore list only include entries with minor differences (like different name or z instead of zoom) or does it also contain entries which are outdated in EII and needs to be fixed there?
follow-up: 13 comment:12 by , 8 years ago
Which lines?
Of the displayed text. It's a simple string compare. E.g. the "name differs" entry consist of a line with URL and two lines with changed data. That's why the count is 3.
Could we please have one more category "updates which need to be solved in EII lists" so we can better distinguish all the entries?
Does the current ignore list only include entries with minor differences (like different name or z instead of zoom) or does it also contain entries which are outdated in EII and needs to be fixed there?
The ignores are ATM all relative to JOSM. Usually that means fixing the entry in EII (or sometimes in JOSM?) would make them obsolete. Where EII uses a TMS instead of the original WMS we can add these as mirror in our config.
follow-up: 17 comment:13 by , 8 years ago
Replying to stoecker:
Which lines?
Of the displayed text. It's a simple string compare. E.g. the "name differs" entry consist of a line with URL and two lines with changed data. That's why the count is 3.
OK, now I understand.
Could we please have one more category "updates which need to be solved in EII lists" so we can better distinguish all the entries?
Does the current ignore list only include entries with minor differences (like different name or z instead of zoom) or does it also contain entries which are outdated in EII and needs to be fixed there?
The ignores are ATM all relative to JOSM. Usually that means fixing the entry in EII (or sometimes in JOSM?) would make them obsolete. Where EII uses a TMS instead of the original WMS we can add these as mirror in our config.
OK, it would be usefull to have 2 ignore lists, one for the entries which are actually correct on both sides (slightly different name or z vs. zoom) and one for the entries which should be fixed at the EII side. This way we can sort the entries without the need to fix them at EII.
Also it would be useful if the ignore lists could be maintained in the wiki. This way only a wiki edit instead of a svn commit is required to update the ignore lists.
follow-up: 18 comment:17 by , 8 years ago
Replying to Klumbumbus:
OK, it would be usefull to have 2 ignore lists, one for the entries which are actually correct on both sides (slightly different name or z vs. zoom) and one for the entries which should be fixed at the EII side. This way we can sort the entries without the need to fix them at EII.
Also it would be useful if the ignore lists could be maintained in the wiki. This way only a wiki edit instead of a svn commit is required to update the ignore lists.
Done.
follow-up: 26 comment:22 by , 8 years ago
Cc: | added |
---|
I don't understand this change. https://josm.openstreetmap.de/wiki/Maps/Estonia?action=diff&version=5 Isn't Mercator the "normal" projection? The previous URLs with -geo
support EPSG4326 and work in Merkator. (I checked for Estonia Basemap (Maaamet))
comment:23 by , 8 years ago
If https is available should that be preferred over http? (e.g. + OpenStreetMap Carto (Standard layer) - https://{switch:a,b,c}.tile.openstreetmap.org/{zoom}/{x}/{y}.png
)
comment:26 by , 8 years ago
Replying to Klumbumbus:
I don't understand this change. https://josm.openstreetmap.de/wiki/Maps/Estonia?action=diff&version=5 Isn't Mercator the "normal" projection? The previous URLs with
-geo
support EPSG4326 and work in Merkator. (I checked for Estonia Basemap (Maaamet))
I changed it because displaying stretched WGS 84 tiles on a Mercator map is a bit of a hack.
Nevertheless, it works fine for this server, so feel free to switch back.
follow-up: 29 comment:28 by , 8 years ago
Thanks a lot Klumbumbus for the cleanups!
JOSM's side of fixes is now (mainly?) done for the moment, so #12313 could be reconsidered. I'm still not really convinced of the "best" flag, but I can see some benefits. Opinions?
And also maybe we could improve the check scripts to detect differences in the polygons between EII and JOSM (and glitches in our own database like unclosed rings) or even other values?
Also now we could add a build check for red entries in the list. Should we?
comment:29 by , 8 years ago
Replying to stoecker:
Thanks a lot Klumbumbus for the cleanups!
JOSM's side of fixes is now (mainly?) done for the moment,
I checked all blue entries if they still deliver tiles and found some, which don't. I will post the list in an own ticket for review.
so #12313 could be reconsidered. I'm still not really convinced of the "best" flag, but I can see some benefits. Opinions?
I also think it's not the best concept. However it would be good to have this flag in our database too, the reasons given in #12313 are valid . Maybe in JOSM we could display a little star in front of the imagery name when it has the best flag.
And also maybe we could improve the check scripts to detect differences in the polygons between EII and JOSM (and glitches in our own database like unclosed rings) or even other values?
Would be good.
Also now we could add a build check for red entries in the list. Should we?
Would be good too. However only for the current checks. Possible polygon check should not yet be included. I guess fixing the polygons will be a longer task.
follow-up: 32 comment:31 by , 8 years ago
Uh, thats a lot. Is there a way to automatically fix these 236 unclosed JOSM shapes by a script? Doing this by hand looks like a lot of boring work.
There seems to be an error in the shape comparison. E.g. imagico.de OSM images for mapping: Eastern Devon Island coast
(Canada) is reported at ImageryCompare, but it has the same shape in maps and in the ELI xml output (in the ELI source file there are some more decimal places, though).
follow-up: 34 comment:32 by , 8 years ago
Replying to Klumbumbus:
Uh, thats a lot. Is there a way to automatically fix these 236 unclosed JOSM shapes by a script? Doing this by hand looks like a lot of boring work.
That would be probably more work that manual. :-) I started yesterday but overlooked that the list is unsorted. I was so happy that I fixed 3 whole countries... The fix is usually simple, copy the first point to the end as well.
There seems to be an error in the shape comparison. E.g.
imagico.de OSM images for mapping: Eastern Devon Island coast
(Canada) is reported at ImageryCompare, but it has the same shape in maps and in the ELI xml output (in the ELI source file there are some more decimal places, though).
There is no approximate check. It checks for equality, so more decimal places means "not equal".
comment:34 by , 8 years ago
Replying to stoecker:
There is no approximate check. It checks for equality, so more decimal places means "not equal".
I thought you use the ELI XML output for the comparison, but you use the json file. The ELI XML file (where I took some shapes from, e.g. all the imagico entries) cuts the last decimal places. So thats the reason for the difference.
BTW geojson is now the default format on ELI and json only legacy. I don't know if it makes sense to change our script.
comment:37 by , 8 years ago
For the coordinates we probably need a better equals() function taking a certain epsilon into account :-)
follow-up: 40 comment:38 by , 8 years ago
When does it report No EII shape:
? When there are only 4 different points?
comment:40 by , 8 years ago
Now it's much better - less reports and better details.
Also removed most of the unclosed polygons.
Replying to Klumbumbus:
When does it report
No EII shape:
? When there are only 4 different points?
I added a special detection for this bounding box similar shapes. Maybe that's wrong. Which entry?
follow-up: 45 comment:44 by , 8 years ago
Hmm, is it useful to add a check for the remaining keyword elements? Polygons have been less work than I expected it to be. I thought it would be a nightmare.
comment:45 by , 8 years ago
Replying to stoecker:
Hmm, is it useful to add a check for the remaining keyword elements?
icon
andprojectons
would be usefulid
seems not useful- maybe a check for the josm database if
bounds
andshape
fit together - copyright/attribution information seems important but I think it is not really possible atm to compare them because ELI has only one property (
attribution
) while in JOSM we have two (attribution-url
andterms-of-use-url
) (even three if you count alsoeula
). These two are necessary for some sources like the opencyclemap http://thunderforest.com/terms/
Attribution must be given to both “Thunderforest” and “OpenStreetMap contributors”.
Users of your site/application must have a working links to www.thunderforest.com and www.openstreetmap.org/copyright
comment:46 by , 8 years ago
Also a check if <id>
is present would be useful. Some weeks ago I fixed an entry without id.
comment:50 by , 8 years ago
@Vincent: Can you add this to automatic checks testing for "red color" in the output?
comment:53 by , 8 years ago
Resolution: | → fixed |
---|---|
Status: | new → closed |
The "best" handling is a separate ticket, so I consider this done.
Milestone renamed