Add bounding box support to WMS/TMS URL specification; Change imagery source format from csv to xml

[stoecker]

Some services are regional only and we should add bounding box as new parameter to take this into account.

[stoecker]

Also attribution data should be added, so it can be displayed properly.

[stoecker]

See r4195, but still missing a proper way to display bounding box to users in imagery preferences.

[bastiK]

This csv format sucks, we should change to xml, like http://git.openstreetmap.org/potlatch2.git/blob/refs/heads/master:/resources/imagery.xml.

People will come and want to add all kinds of information, like EULA, different forms of attribution and so on. This isn't easy to encode in a semicolon separated list (unless we add key:value notation). So the original simplicity gets lost.

[stoecker]

I know, but it was much easier to extend than to define an totally new format. :-)

Do you want to improve it? Nevertheless we already have a different URL format than potlatch, so 100% compatibility is hard (probably needing another tag than <url>, so both styles can be defined in one file).

[bastiK]

Yes, I'll do something. It's probably not easy/worthwhile to get full compatibility.

For completeness: http://gitorious.org/merkaartor/main/blobs/master/share/TmsServersList.xml and http://gitorious.org/merkaartor/main/blobs/master/share/WmsServersList.xml

We have to get the following right:

• some attributes are mandatory, like EULA or attribution. JOSM should ignore the entire entry if it doesn't support it.
• other things, like bounding box or zoom limits can be ignored without problem

This gets interesting, when old JOSM versions are still in use, but new features are added to the format.

[stoecker]

Ok, collecting some links regarding data formats:

Yours:

• ​Potlatch XML
• ​Merkaartor WMS XML
• ​Merkaartor TMS XML

• ​Glopus XML
• ​Locus XML
• ​JustMaps XML
• ​Mapviewer Custom code, but interesting format to know what parameters should be supported.
• ​GlobalScout Similar parameter format to ours.

[stoecker]

One more:

• ​QLandkarteGT WMS XML

[bastiK]

xml format for imagery sources (first draft)

[bastiK]

Here is a preliminary patch. Suggestions for improvement?

How shall we do the transition? Simply keep the site Maps as long as necessary and create a new site Imagery? Nice table, by the way. :)

[bastiK]

forgot to add

[stoecker]

Some comments from short look:

• Reader should support both new and old format (autodetecting)
• We simply can change the Maps page to use XML style. The file maps has already a backwards compatibility and could simply deliver old style files for older JOSM versions.
• "src/org/openstreetmap/josm/io/imagery/OsmosnimkiOffsetServer.java" should not use extended URL, this is a bug in current code. At least the zoom specs should be stripped for it.

[stoecker]

Another note: I would not want to support the automatic add "/{zoom}/{x}/{y}.png" for the new file. This should be explicit. Same for WMS, this should also support the {...} style always.

[bastiK]

Replying to stoecker:

Some comments from short look:

• Reader should support both new and old format (autodetecting)
• We simply can change the Maps page to use XML style. The file maps has already a backwards compatibility and could simply deliver old style files for older JOSM versions.

Is the old format used anywhere, except on Maps? Would be easier to make a clear cut. And you wouldn't need a script to convert the xml to csv. :)

Another note: I would not want to support the automatic add "/{zoom}/{x}/{y}.png" for the new file. This should be explicit. Same for WMS, this should also support the {...} style always.

Agreed, I won't support the "osm" shortcut for attribution either. A bit of copy & paste is better than too much magic.

[stoecker]

Hmm, while a clear cut surely has some advantages I'm usually more for smooth transition. So I would anyway make a XML-->CSV conversion. But you are right, there is no reason to support this in JOSM itself.

The remaining question is how we want to store the data in preferences. I would vote instead of copying the data we could separate it into self-made URLs and standard URLs and for standard URL we only store the URL and take the other data from the info file always (much like the silent attribution copy I implemented).

[bastiK]

Replying to stoecker:

Hmm, while a clear cut surely has some advantages I'm usually more for smooth transition. So I would anyway make a XML-->CSV conversion. But you are right, there is no reason to support this in JOSM itself.

Too late, I've added auto-detection, here is the latest version: ​https://github.com/bastik/josm/compare/imagery-xml

I'm a little unsure about wms at the moment: A capabilities request returns a (potentially long) list of supported SRS. Some have EPSG:4326 (WSG84), but not EPSG:3857 (900913).

The problem is, that the current user projection is not known in advance. It could even come from a plugin.

So we could either make a capabilities request whenever the background is activated or we save the list of possible projections in the xml preferences, like so:

    <entry>
        <name>Landsat</name>
        <type>wms</type>
        <default>true</default>
        <url><![CDATA[http://irs.gis-lab.info/?layers=landsat&bbox={s},{w},{n},{e}&width={width}&heigth={height}&srs={srs}]]></url>
        <supported-srs>
            <srs>NASA:4326</srs>
            <srs>EPSG:32641</srs>
            <srs>EPSG:3857</srs>
            <srs>EPSG:3785</srs>
            <srs>EPSG:32642</srs>
            <srs>EPSG:32640</srs>
            <srs>EPSG:4326</srs>
            <srs>EPSG:900913</srs>
            <srs>EPSG:3395</srs>
            <srs>EPSG:32639</srs>
            <srs>EPSG:32638</srs>
            <srs>EPSG:32637</srs>
            <srs>EPSG:32636</srs>
            <srs>EPSG:32635</srs>
        </supported-srs>
        <default-srs>EPSG:3857</default-srs>
    </entry>

It's a bit verbose, though. But for the automatic retrieval you also need the correct service url, so the list of SRS might be more straightforward.

The remaining question is how we want to store the data in preferences. I would vote instead of copying the data we could separate it into self-made URLs and standard URLs and for standard URL we only store the URL and take the other data from the info file always (much like the silent attribution copy I implemented).

For the custom imagery sources we should offer the same features as for the standard ones, e.g. maxzoom, attribution and so on. So if we'd have a common preference format, this would be more consistent. I think with the ListOfStructs Preference type I've added a few months ago, this should work quite well.

[stoecker]

I would add all supported SRS in file as well. Our GetCapabilities is not really trustworthy at the moment.

I will care for the server/wiki support this week. Until then I think you could checkin the parsing code to SVN, but don't add the XML or change the link.

A note: For html: you should also switch to {...} style and no auto URL.

Is there a reason why you use CDATA for URL?

[bastiK]

In [4240/josm]:

see #6532 - add support for xml imagery source format

(todo: the specified server projections aren't used properly in WMSGrabber)

[bastiK]

Replying to stoecker:

I would add all supported SRS in file as well. Our GetCapabilities is not really trustworthy at the moment.

I will care for the server/wiki support this week. Until then I think you could checkin the parsing code to SVN, but don't add the XML or change the link.

A note: For html: you should also switch to {...} style and no auto URL.

We can still finish the switch to the verbose style later.

Is there a reason why you use CDATA for URL?

Yes, avoid escaping of & for easier copy & paste.

[bastiK]

Who's turn is it?

[stoecker]

Sorry, had not yet time (I did get hold by the I18N problem xeen brought up). I try to catch up with this issue as soon as possible.

[bastiK]

Switch to xml format is finished. Shall we open a new ticket for bounding box visualization?

[stoecker]

I think #6740 can be used as "bounding box" ticket.