Modify

Opened 4 months ago

Closed 7 weeks ago

Last modified 6 weeks ago

#18440 closed enhancement (fixed)

Heads up: Maxar implementing new anti-scraping techniques

Reported by: Stereo Owned by: Don-vip
Priority: normal Milestone: 20.02
Component: Core imagery Version:
Keywords: maxar api key Cc: sommerluk

Attachments (0)

Change History (33)

comment:1 Changed 4 months ago by Boggedy

This may also require an auto expire feature for the JOSM imagery cache where Maxar and ESRI tiles are all autodeleted on exit.

comment:2 Changed 3 months ago by sommerluk

Cc: sommerluk added

comment:3 Changed 3 months ago by Don-vip

Ticket #18473 has been marked as a duplicate of this ticket.

comment:4 Changed 3 months ago by Don-vip

Keywords: maxar added

comment:5 Changed 3 months ago by Don-vip

Ticket #18516 has been marked as a duplicate of this ticket.

comment:6 Changed 3 months ago by Don-vip

See https://josm.openstreetmap.de/ticket/18473#comment:3 comments about the removal from default entries

comment:7 Changed 8 weeks ago by Don-vip

Milestone: 20.02
Owner: changed from team to Don-vip
Status: newassigned

comment:8 Changed 8 weeks ago by Don-vip

I've received an e-mail from Maxar providing us (JOSM) a new API key to restore access to Maxar imagery. Their only condition is to not make the API key public on the wiki or the source code.

I already began related work in #17744 / [o35017] when the Bing API key was unavailable. I'll work on this tomorrow during GeoFabrik hack week-end.

comment:9 Changed 8 weeks ago by stoecker

If the key should not be in the wiki or the source, where should it be?

comment:10 Changed 8 weeks ago by Don-vip

I was thinking about providing it at build time and storing it somewhat in JOSM jar. Not completely hidden, but at least not easily crawlable. They seem ok with this.

comment:11 Changed 7 weeks ago by stoecker

That's not possible. It would be a license violation of JOSM GPL.

comment:12 Changed 7 weeks ago by Don-vip

Ok. Does it work if JOSM retrieves the key from JOSM website using a new endpoint? I'm thinking of a POST request with a user-agent check.

comment:13 Changed 7 weeks ago by stoecker

That would work.

comment:14 Changed 7 weeks ago by stoecker

If so, please make it expandable to other JOSM specific keys.

comment:15 Changed 7 weeks ago by Don-vip

In 15853/josm:

see #18440 - non-blocking warning in case of invalid TMS entries

comment:16 Changed 7 weeks ago by Don-vip

JMapViewer updated in [o35320].

comment:17 in reply to:  14 ; Changed 7 weeks ago by Don-vip

Replying to stoecker:

If so, please make it expandable to other JOSM specific keys.

What other keys do you have in mind?

comment:18 in reply to:  17 ; Changed 7 weeks ago by stoecker

Replying to Don-vip:

Replying to stoecker:

If so, please make it expandable to other JOSM specific keys.

What other keys do you have in mind?

We have a few other JOSM specific map keys. If we setup such a feature it should be possible to support other services as well. It should not be Maxar only.

comment:19 Changed 7 weeks ago by stoecker

E.g.

https://josm.openstreetmap.org/mapkey?id=maxar

returns the maxar key and ...

comment:20 Changed 7 weeks ago by SimonPoole

@stoecker the GPL in general does not apply to configuration.

comment:21 in reply to:  20 ; Changed 7 weeks ago by stoecker

Replying to SimonPoole:

@stoecker the GPL in general does not apply to configuration.

That's not configuration. GPL applies to sources and the tools/toolchain to build an executable from this. So there cannot be any secret in the build process. The code signing we do for JOSM is already a border case.

comment:22 in reply to:  18 Changed 7 weeks ago by Don-vip

Replying to stoecker:

If we setup such a feature it should be possible to support other services as well. It should not be Maxar only.

No problem, this is what I had in mind.
EDIT: These are the entries I found with an API key:

  • Bing
  • City_of_Melbourne_Feb_2019
  • City_of_Melbourne_May_2018
  • geoimage.at
  • geovekst-nib
  • IBGE_DF_Addresses
  • IBGE_Nomes_Ruas
  • IBGE_Salvador_Streets
  • lantmateriet-orto1960
  • lantmateriet-orto1975
  • lantmateriet-topowebb
  • LINZ_NZ_Aerial_Imagery
  • LINZ_NZ_Topo50_Gridless_Maps
  • Mapbox
  • mapbox_locator_overlay
  • Maxar-Premium
  • Maxar-Standard
  • New_and_Misaligned_TIGER_Roads-2013
  • opencylemap
  • tf-landscape
  • US_Forest_Service_roads_overlay
  • US-TIGER-Roads-2017
  • US-TIGER-Roads-2018
  • US-TIGER-Roads-2019
  • WSDOT-Roads-2017

Any missing?

Last edited 7 weeks ago by Don-vip (previous) (diff)

comment:23 Changed 7 weeks ago by Don-vip

In 15855/josm:

see #18440 - Fetch imagery API keys from JOSM website. Restore access to Maxar imagery. Requires JMapViewer 2.13

comment:24 in reply to:  21 Changed 7 weeks ago by anonymous

Replying to stoecker:

Replying to SimonPoole:

@stoecker the GPL in general does not apply to configuration.

That's not configuration. GPL applies to sources and the tools/toolchain to build an executable from this. So there cannot be any secret in the build process. The code signing we do for JOSM is already a border case.

This not even the interpretation of the FSF.

comment:25 Changed 7 weeks ago by stoecker

The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable.

I don't care about any "interpretation", especially given by an anonymous commenter. I follow the GPL texts and these are very clear in this fact.

Last edited 7 weeks ago by stoecker (previous) (diff)

comment:26 Changed 7 weeks ago by Don-vip

In 15868/josm:

see #18440 - cache API keys in memory

comment:27 Changed 7 weeks ago by Don-vip

Keywords: api key added
Resolution: fixed
Status: assignedclosed

Done for Maxar. Follow-up in #18726

comment:28 Changed 7 weeks ago by Don-vip

In 15871/josm:

see #18440 - register API key provider in integration test

comment:29 Changed 7 weeks ago by stoecker

Could you change the pattern to lowercase {apikey}? that's not a Java variable name.

comment:30 in reply to:  29 Changed 7 weeks ago by Don-vip

Replying to stoecker:

Could you change the pattern to lowercase {apikey}? that's not a Java variable name.

Done in [o35329].

comment:31 Changed 6 weeks ago by Klumbumbus

There are at least two ways to display the connectId in JOSM in cleartext. Is that a problem?

comment:32 in reply to:  31 Changed 6 weeks ago by stoecker

Replying to Klumbumbus:

There are at least two ways to display the connectId in JOSM in cleartext. Is that a problem?

Hopefully not. Current solution is the best we can do in an OpenSource software.

comment:33 in reply to:  31 Changed 6 weeks ago by Don-vip

Replying to Klumbumbus:

There are at least two ways to display the connectId in JOSM in cleartext. Is that a problem?

No, I warned them about this and they're ok. They just don't want the key to be too easily scraped (using a search engine for example).

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Don-vip.
as The resolution will be set.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.