Modify

Opened 6 years ago

Last modified 3 years ago

#16258 new enhancement

GDPR related API changes

Reported by: Don-vip Owned by: team
Priority: major Milestone:
Component: Core Version:
Keywords: GDPR api anonymous Cc: Stereo

Description (last modified by Don-vip)

Seen on dev mailing list:

The LWG has made some recommendations about what we need to change on the web site and API to comply with future European data protection rules. On the whole this boils down to "logged-in users get the same stuff they get today, but guests who are not logged in will not see details about users".

The detailed LWG recommendations are here.

List of what Frederik believes needs to be changed on the API and web site, here.

For JOSM it means we have to check the various anonymous API calls don't provoke any crash when data is missing. I doubt it :)

GDPR enters into force on 25th May. So we must release 18.05 before that.

The API is not going to change before GDPR enters in to force, as per Frederik's e-mail:

the general feeling is that it's going to be something like a 6 month project for the whole package, with some measures certainly being taken early on to demonstrate our will to comply. Regarding the API changes specifically, once we're clear about the scope we'll need to determine how much work it is to make the changes, and then find someone to make them... it's certainly on a scale of months not weeks.

Attachments (0)

Change History (17)

comment:1 by Don-vip, 6 years ago

Description: modified (diff)
Milestone: 18.0518.07

comment:2 by Don-vip, 6 years ago

Description: modified (diff)

comment:3 by stoecker, 6 years ago

Would be nice when we have some time between these changes in DEV API and the real one. Also for JOSM this probably will be a "few months" topic.

To ease changes we may keep internally the structure and simply replace them with a "pseudo" user on import. Then only import/export and the external calls (history, userinfo, webpage calls) must handle these cases. Not all the dialogs.

comment:4 by mmd, 6 years ago

Most JOSM users should already have entered some credentials, and as long as you provide those credentials to the API, you're still getting exactly the same data as before. Only for unauthenticated API calls some data will get dropped, like mentioned on the GDPR Wiki page.

JOSM currently offers both OAuth and Basic auth. Unfortunately, for those calls handled by cgimap (most importantly that's /map), Basic auth is not yet implemented. If you think that's an important use case for JOSM users, please raise an issue over at https://github.com/zerebubuth/openstreetmap-cgimap/issues and suggest to have Basic Auth also implemented for cgimap.

comment:5 by mmd, 6 years ago

I'm just testing a bit on a custom /map implementation that hides changeset, uid and user for unauthenticated users. It's meant as a very first impression where potential issues might be, not much more.

Object history

Missing changeset attribute

org.openstreetmap.josm.io.OsmTransferException: org.xml.sax.SAXException: (3,83)Notwendiges Attribut "changeset" fehlt.
	at org.openstreetmap.josm.io.OsmServerHistoryReader.parseHistory(OsmServerHistoryReader.java:77)
	at org.openstreetmap.josm.gui.history.HistoryLoadTask.loadHistory(HistoryLoadTask.java:197)
	at org.openstreetmap.josm.gui.history.HistoryLoadTask.loadHistory(HistoryLoadTask.java:187)
	at org.openstreetmap.josm.gui.history.HistoryLoadTask.realRun(HistoryLoadTask.java:172)
	at org.openstreetmap.josm.gui.PleaseWaitRunnable.doRealRun(PleaseWaitRunnable.java:95)
	at org.openstreetmap.josm.gui.PleaseWaitRunnable.run(PleaseWaitRunnable.java:143)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.xml.sax.SAXException: (3,83)Notwendiges Attribut "changeset" fehlt.
	at org.openstreetmap.josm.io.OsmHistoryReader$Parser.throwException(OsmHistoryReader.java:48)
	at org.openstreetmap.josm.io.AbstractParser.getMandatoryAttributeLong(AbstractParser.java:44)
	at org.openstreetmap.josm.io.AbstractParser.createPrimitive(AbstractParser.java:110)
	at org.openstreetmap.josm.io.AbstractParser.startWay(AbstractParser.java:149)
	at org.openstreetmap.josm.io.AbstractParser.doStartElement(AbstractParser.java:187)
	at org.openstreetmap.josm.io.OsmHistoryReader$Parser.startElement(OsmHistoryReader.java:58)
	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:509)
	at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:380)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2787)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606)
	at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:118)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777)
	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
	at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
	at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:327)
	at org.openstreetmap.josm.tools.Utils.parseSafeSAX(Utils.java:1418)
	at org.openstreetmap.josm.io.OsmHistoryReader.parse(OsmHistoryReader.java:94)
	at org.openstreetmap.josm.io.OsmServerHistoryReader.parseHistory(OsmServerHistoryReader.java:70)
	... 10 more

Map result:

<?xml version='1.0' encoding='UTF-8'?>
<osm version='0.6' generator='custom'>
  <bounds minlat='39.9105262' minlon='116.3581486' maxlat='39.9140166' maxlon='116.363392' origin='CGImap 0.6.0 (3940 ubuntu)' />
  <node id='-39436' action='modify' lat='39.91321943402' lon='116.35923758974' />
  <node id='5001648038' timestamp='2018-05-06T21:05:31Z' version='1' lat='39.9129772' lon='116.3657367' />
  <node id='5001648039' timestamp='2018-05-06T21:05:31Z' version='1' lat='39.9129375' lon='116.3632287' />
  <node id='5001648040' timestamp='2018-05-06T21:05:31Z' version='1' lat='39.9136042' lon='116.3654369' />
  <node id='5001648041' timestamp='2018-05-06T21:05:31Z' version='1' lat='39.9118641' lon='116.3657018' />
...
Last edited 6 years ago by Don-vip (previous) (diff)

in reply to:  5 comment:6 by Don-vip, 6 years ago

Replying to mmd:

I'm just testing a bit on a custom /map implementation that hides changeset, uid and user for unauthenticated users. It's meant as a very first impression where potential issues might be, not much more.

Thank you :) Is it available somewhere so that we could test ourselves?

comment:7 by Don-vip, 6 years ago

In 13762/josm:

see #16258 - changeset attribute can now be missing

comment:8 by mmd, 6 years ago

Great, thanks a lot for the fix!

Is it available somewhere so that we could test ourselves?

At this time, I'm testing a local prototype, along with a local DB and Rails port. Hopefully, we'll have something more accessible soon... see https://github.com/zerebubuth/openstreetmap-cgimap/issues/144

You could also try to create an osm.xml file locally where those attributes are missing, and load it into JOSM. In any case, a proper API implementation would be more helpful, though.

comment:9 by Don-vip, 6 years ago

OK, let us know once the prototype is available.

comment:10 by Don-vip, 6 years ago

Milestone: 18.0718.08

comment:11 by Don-vip, 6 years ago

See #16499 for GDPR-compliant extracts

comment:12 by Don-vip, 6 years ago

Milestone: 18.08

comment:15 by simon04, 3 years ago

Anything left to be done for JOSM?

comment:16 by Don-vip, 3 years ago

Cc: Stereo added

It depends if GDPR changes are still on OSMF board's radar. It seems so, see

https://wiki.osmfoundation.org/wiki/Board/Minutes/2020-04

Mikel (Taken up 2020-02) (was: Frederik/Heather) 2019-05-18 to complete and share job description for someone to prepare a pull request for acceptance against the current “openstreetmap-website” code that will implement these changes (related blogpost).

# Mikel plans (2020-02) to circulate to people in Ruby community

F2F 2019 GDPR

https://wiki.osmfoundation.org/wiki/Board/Minutes/2020-08

We also need a representative in the EU for GDPR purposes.
Guillaume plans to have meeting with EU lawyer.

@Stereo any news from expected GDPR changes that might impact us?

Last edited 3 years ago by Don-vip (previous) (diff)

comment:17 by Stereo, 3 years ago

No news for now, but I'll update this ticket if and when anything comes.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as new The owner will remain team.
as The resolution will be set. Next status will be 'closed'.
to The owner will be changed from team to the specified user.
Next status will be 'needinfo'. The owner will be changed from team to Don-vip.
as duplicate The resolution will be set to duplicate. Next status will be 'closed'. The specified ticket will be cross-referenced with this ticket.
The owner will be changed from team to anonymous. Next status will be 'assigned'.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.