Modify

Opened 12 months ago

Closed 10 months ago

Last modified 9 months ago

#18440 closed enhancement (fixed)

Heads up: Maxar implementing new anti-scraping techniques

Reported by: Stereo Owned by: Don-vip
Priority: normal Milestone: 20.02
Component: Core imagery Version:
Keywords: maxar api key Cc: sommerluk

Attachments (0)

Change History (33)

comment:1 Changed 12 months ago by Boggedy

This may also require an auto expire feature for the JOSM imagery cache where Maxar and ESRI tiles are all autodeleted on exit.

comment:2 Changed 12 months ago by sommerluk

Cc: sommerluk added

comment:3 Changed 11 months ago by Don-vip

Ticket #18473 has been marked as a duplicate of this ticket.

comment:4 Changed 11 months ago by Don-vip

Keywords: maxar added

comment:5 Changed 11 months ago by Don-vip

Ticket #18516 has been marked as a duplicate of this ticket.

comment:6 Changed 11 months ago by Don-vip

See https://josm.openstreetmap.de/ticket/18473#comment:3 comments about the removal from default entries

comment:7 Changed 10 months ago by Don-vip

Milestone: 20.02
Owner: changed from team to Don-vip
Status: newassigned

comment:8 Changed 10 months ago by Don-vip

I've received an e-mail from Maxar providing us (JOSM) a new API key to restore access to Maxar imagery. Their only condition is to not make the API key public on the wiki or the source code.

I already began related work in #17744 / [o35017] when the Bing API key was unavailable. I'll work on this tomorrow during GeoFabrik hack week-end.

comment:9 Changed 10 months ago by stoecker

If the key should not be in the wiki or the source, where should it be?

comment:10 Changed 10 months ago by Don-vip

I was thinking about providing it at build time and storing it somewhat in JOSM jar. Not completely hidden, but at least not easily crawlable. They seem ok with this.

comment:11 Changed 10 months ago by stoecker

That's not possible. It would be a license violation of JOSM GPL.

comment:12 Changed 10 months ago by Don-vip

Ok. Does it work if JOSM retrieves the key from JOSM website using a new endpoint? I'm thinking of a POST request with a user-agent check.

comment:13 Changed 10 months ago by stoecker

That would work.

comment:14 Changed 10 months ago by stoecker

If so, please make it expandable to other JOSM specific keys.

comment:15 Changed 10 months ago by Don-vip

In 15853/josm:

see #18440 - non-blocking warning in case of invalid TMS entries

comment:16 Changed 10 months ago by Don-vip

JMapViewer updated in [o35320].

comment:17 in reply to:  14 ; Changed 10 months ago by Don-vip

Replying to stoecker:

If so, please make it expandable to other JOSM specific keys.

What other keys do you have in mind?

comment:18 in reply to:  17 ; Changed 10 months ago by stoecker

Replying to Don-vip:

Replying to stoecker:

If so, please make it expandable to other JOSM specific keys.

What other keys do you have in mind?

We have a few other JOSM specific map keys. If we setup such a feature it should be possible to support other services as well. It should not be Maxar only.

comment:19 Changed 10 months ago by stoecker

E.g.

https://josm.openstreetmap.org/mapkey?id=maxar

returns the maxar key and ...

comment:20 Changed 10 months ago by SimonPoole

@stoecker the GPL in general does not apply to configuration.

comment:21 in reply to:  20 ; Changed 10 months ago by stoecker

Replying to SimonPoole:

@stoecker the GPL in general does not apply to configuration.

That's not configuration. GPL applies to sources and the tools/toolchain to build an executable from this. So there cannot be any secret in the build process. The code signing we do for JOSM is already a border case.

comment:22 in reply to:  18 Changed 10 months ago by Don-vip

Replying to stoecker:

If we setup such a feature it should be possible to support other services as well. It should not be Maxar only.

No problem, this is what I had in mind.
EDIT: These are the entries I found with an API key:

  • Bing
  • City_of_Melbourne_Feb_2019
  • City_of_Melbourne_May_2018
  • geoimage.at
  • geovekst-nib
  • IBGE_DF_Addresses
  • IBGE_Nomes_Ruas
  • IBGE_Salvador_Streets
  • lantmateriet-orto1960
  • lantmateriet-orto1975
  • lantmateriet-topowebb
  • LINZ_NZ_Aerial_Imagery
  • LINZ_NZ_Topo50_Gridless_Maps
  • Mapbox
  • mapbox_locator_overlay
  • Maxar-Premium
  • Maxar-Standard
  • New_and_Misaligned_TIGER_Roads-2013
  • opencylemap
  • tf-landscape
  • US_Forest_Service_roads_overlay
  • US-TIGER-Roads-2017
  • US-TIGER-Roads-2018
  • US-TIGER-Roads-2019
  • WSDOT-Roads-2017

Any missing?

Last edited 10 months ago by Don-vip (previous) (diff)

comment:23 Changed 10 months ago by Don-vip

In 15855/josm:

see #18440 - Fetch imagery API keys from JOSM website. Restore access to Maxar imagery. Requires JMapViewer 2.13

comment:24 in reply to:  21 Changed 10 months ago by anonymous

Replying to stoecker:

Replying to SimonPoole:

@stoecker the GPL in general does not apply to configuration.

That's not configuration. GPL applies to sources and the tools/toolchain to build an executable from this. So there cannot be any secret in the build process. The code signing we do for JOSM is already a border case.

This not even the interpretation of the FSF.

comment:25 Changed 10 months ago by stoecker

The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable.

I don't care about any "interpretation", especially given by an anonymous commenter. I follow the GPL texts and these are very clear in this fact.

Last edited 10 months ago by stoecker (previous) (diff)

comment:26 Changed 10 months ago by Don-vip

In 15868/josm:

see #18440 - cache API keys in memory

comment:27 Changed 10 months ago by Don-vip

Keywords: api key added
Resolution: fixed
Status: assignedclosed

Done for Maxar. Follow-up in #18726

comment:28 Changed 10 months ago by Don-vip

In 15871/josm:

see #18440 - register API key provider in integration test

comment:29 Changed 10 months ago by stoecker

Could you change the pattern to lowercase {apikey}? that's not a Java variable name.

comment:30 in reply to:  29 Changed 10 months ago by Don-vip

Replying to stoecker:

Could you change the pattern to lowercase {apikey}? that's not a Java variable name.

Done in [o35329].

comment:31 Changed 9 months ago by Klumbumbus

There are at least two ways to display the connectId in JOSM in cleartext. Is that a problem?

comment:32 in reply to:  31 Changed 9 months ago by stoecker

Replying to Klumbumbus:

There are at least two ways to display the connectId in JOSM in cleartext. Is that a problem?

Hopefully not. Current solution is the best we can do in an OpenSource software.

comment:33 in reply to:  31 Changed 9 months ago by Don-vip

Replying to Klumbumbus:

There are at least two ways to display the connectId in JOSM in cleartext. Is that a problem?

No, I warned them about this and they're ok. They just don't want the key to be too easily scraped (using a search engine for example).

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Don-vip.
as The resolution will be set.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.