Modify

Opened 10 months ago

Last modified 5 weeks ago

#16100 new enhancement

Strava Global Heatmap in JOSM

Reported by: anonymous Owned by: allier-osm
Priority: normal Milestone:
Component: Plugin strava_heatmap Version:
Keywords: strava heatmap oauth authentication Cc: pschonmann

Description

With the latest changes to the Global Heatmap, street-level detail is only available when logged into a Strava account. A result of this is unfortunately that the Heatmap has become a more or less useless tool for improving Openstreetmap using JOSM. I guess that this was an unintended result of the changes made to the Heatmap.

Attachments (0)

Change History (31)

comment:1 Changed 10 months ago by stoecker

Owner: changed from team to anonymous
Status: newneedinfo

Well. You did not say what you want.

comment:2 Changed 10 months ago by anonymous

I just want it fixed... I guess this could either be done on the Strava side of things, or by adding the possibility in JOSM to log into the Strava account.

comment:3 Changed 10 months ago by Wegspotter

Same problem here. Can this be fixed?

comment:4 Changed 10 months ago by Don-vip

Keywords: strava heatmap added

comment:5 Changed 10 months ago by Don-vip

iD issue closed as wontfix: https://github.com/openstreetmap/iD/issues/4896

There was doubts about the possibility to use Strava for editing OSM, but this has been confirmed by a Strava employee here: https://github.com/osmlab/editor-layer-index/pull/373#issuecomment-352544421

OSM wiki: https://wiki.openstreetmap.org/wiki/Strava#Global_Heatmap

This would probably require to setup OAuth authentication like with OSM API, see https://developers.strava.com/docs/authentication/

It would be nice to make it generic: define in Maps that a layer require OAuth authentication and provide authorization page URL. This way we wouldn't have Strava-specific code.

comment:6 Changed 10 months ago by Don-vip

Keywords: oauth authentication added
Type: defectenhancement

comment:7 Changed 10 months ago by Don-vip

Owner: changed from anonymous to team
Status: needinfonew

comment:8 Changed 10 months ago by alpterlab

exactly!Since the new Heat Map is being developed underMapbox GL they just need to set proper individual access tokens in order to associate requests to API resources with each personal STRAVA account. I guess... but... How long would it take ?

comment:9 in reply to:  5 Changed 10 months ago by stoecker

Replying to Don-vip:

This would probably require to setup OAuth authentication like with OSM API, see https://developers.strava.com/docs/authentication/

It would be nice to make it generic: define in Maps that a layer require OAuth authentication and provide authorization page URL. This way we wouldn't have Strava-specific code.

It also requires users to understand how OAuth works and to setup OAuth permissions. For OSM that worked not well until the "Fully Automatic" approach, which is site specific.

comment:10 Changed 10 months ago by Klumbumbus

Possible relevant: The mapillary plugin uses a login system too.

comment:13 in reply to:  12 ; Changed 9 months ago by anonymous

Replying to solitone:
This works for me: tms[3,17]:https://heatmap-external-{switch:a,b,c}.strava.com/tiles-auth/both/bluered/{zoom}/{x}/{y}.png?Key-Pair-Id=MYVALUE&Signature=MYVALUE&Policy=MYVALUE where MYVALUE are the respective cookies on your PC value, but the zoom level is the same as before. Of course, you need to have a strava account and you need to be logged in. And once in a while, you have to paste the new cookies values as they expire. But it's worth the effort.

Replying to wegspotter:

Maybe there is something useful there > https://help.openstreetmap.org/questions/62803/strava-heatmap-no-longer-useful-for-osm-mappers-anything-we-can-do?

No luck here with the suggested new URL:
tms[3,17]:http://globalheat.strava.com/tiles/cycling/color1/{zoom}/{x}/{y}.png

comment:14 in reply to:  13 Changed 9 months ago by solitone

Replying to anonymous:

This works for me: tms[3,17]:https://heatmap-external-{switch:a,b,c}.strava.com/tiles-auth/both/bluered/{zoom}/{x}/{y}.png?Key-Pair-Id=MYVALUE&Signature=MYVALUE&Policy=MYVALUE where MYVALUE are the respective cookies on your PC value, but the zoom level is the same as before. Of course, you need to have a strava account and you need to be logged in. And once in a while, you have to paste the new cookies values as they expire. But it's worth the effort.

Hi & thanks for your feedback. Unfortunately I can't make it work. If I use "tms[3,17] ..." I get "Error: no protocol". If I use "tms[3,11] ..." I get a message complaining that there is "no tile at this zoom level".

Thanks again & Regards

Last edited 9 months ago by solitone (previous) (diff)

comment:16 Changed 8 months ago by SHARCRASH

I tried to add the address with my cooky values as a TMS but i still get an error "No protocol... etc". I noticed that the keys of MYVALUEx start actually with CloudFront- in the developer console (Firefox and Chrome). Can anybody help please?

comment:17 in reply to:  16 Changed 8 months ago by solitone

Replying to SHARCRASH:

I tried to add the address with my cooky values as a TMS but i still get an error "No protocol... etc".

Double check that address. I also had issue, and found out that I needed to specify "Key-Pair-Id", "Policy", and "Signature" in this exact same order.

I noticed that the keys of MYVALUEx start actually with CloudFront- in the developer console (Firefox and Chrome). Can anybody help please?

Yes, that's true, but in the URL that you enter you should remove the "CloudFront-" part, and specify only "Key-Pair-Id", "Policy", and "Signature" as keys.

comment:18 Changed 8 months ago by solitone

BTW, to get those three CloudFront-Key-Pair-Id, CloudFront-Policy, and CloudFront-Signature cookies, you need to open the Strava heatmap from your browser--it's not enough to log in Strava.

As explained at [1], the authentication flow is as follows:
1) POST https://www.strava.com/session, email=<STRAVA EMAIL>, password=<STRAVA PASSWORD>
-> Sets _strava4_session cookie
2) GET https://heatmap-external-a.strava.com/auth with session cookie set
-> Sets CloudFront-Signature, CloudFront-Policy, and CloudFront-Key-Pair-Id cookies
3) GET https://heatmap-external-a.strava.com/t ... h/all/hot/{z}/{x}/{y}.png?v=19 with CloudFront cookies set

[1] http://duweis.com/forums/viewtopic.php?f=13&t=761

comment:19 Changed 8 months ago by SHARCRASH

Found the problem! Actually, JOSM adds another TMS in the address: tms:tms[3,15]:https... instead of tms[3,15]:https...
Thank you Sonitone!

comment:20 Changed 8 months ago by jflamy

Still getting "no protocol" error, and "no tiles found at current zoom level" (which is 13).
Proceeded as follows

  • used chrome developer mode to clear my strava cookies
  • logged on to strava global heat map
  • retrieved the three cookies for policy, key-pair-id, and signature
  • followed advice regarding ordering of cookie values when passed explicitly
  • added an imaging entry at the bottom of the imaging preferences dialog, and saved, as follows.
    tms[3,15]:​https://heatmap-external-{switch:a,b,c}.strava.com/tiles-auth/both/bluered/{zoom}/{x}/{y}.png?Key-Pair-Id=APKAIDPUN4QMG7VUQPSA&Policy=eyJTdGF0ZW1lbnQiOiBbeyJSZXNvdXJjZSI6Imh0dHBzOi8vaGVhdG1hcC1leHRlcm5hbC0qLnN0cmF2YS5jb20vKiIsIkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTUyODIyMDUxMX0sIkRhdGVHcmVhdGVyVGhhbiI6eyJBV1M6RXBvY2hUaW1lIjoxNTI2OTk2NTExfX19XX0_&Signature=YHy7BrWO1~5tm~i~6SXxQhvn4VpVKWlulFk70~sznKuOPl8TMG-~CgH4gELwp3N9fUOejLpk7ef4lwBJIAq1~yBbIDqo5pJsOJgAc3Q~tVxpSQVA5ayKMF3fJjRbmtGAlf5EoHG19INFGhNrBnw3ql9H4zeIzMYEFb9M29Y1IFUf6CAsSNBCjt0CUCZoYWXu4UpZWVVnlu~xLRPs6osyEv7qvZOlxe9AP5B~CScSfDmcEDUcUkmD5sPk0CMPGls6kCgHOsUiuh4hvI-beEdjer5UPBb0Y2UOmAIV1x~Qd8a05cpoIsBDmkZSV6ou1eIBXikUnTJOb8VJGC5VYIY9kA__
    
  • checked using the information button that the proper parameters were being used for the layer

Am out of ideas.

comment:21 Changed 8 months ago by solitone

Everything seems fine, but a single character can cause problems. Start up again with the default strava entry and change it so that it considers the three cookies. Make sure you have only one "tms" string (as SHARCRASH pointed out).

comment:22 Changed 5 months ago by csytsma

I finally got it to work, and am now showing high resolution heatmap data in JOSM, just like it used to. I do have a Premium account, which you have to have to get the high resolution data. A plain login will only display low resolution data.

I'm not sure what I did to exactly get it working, as it didn't initially work after following the steps above. Here are the steps I took to walk through the process to confirm my URL and values were correct:

  1. Go to Strava Heatmap (www.strava.com/heatmap) and make sure you are logged in with Premium/Summit account
  2. Confirm you can see high-resolution heatmaps
  3. In Chrome, right-click and Inspect, or just open Developer Tools
  4. Click on the Network tab
  5. Zoom into an area, preferably familiar
  6. In the left Name column, look for files named something like 1432.png?v=19
  7. Click on the png file
  8. Click on the Preview tab, and you should see the heatmap image on the right
  9. Right-click on the image, and Copy Image Url
  10. Open new tab and paste into URL. Confirm that the image loads, so you know you have a good URL.
  11. Put the 3 cookie values (Key-Pair-Id, Signature, Policy) on to the end of your url, just like described above. Confirm that you can still view the image with the new URL.
  12. Open a new browser that doesn't have the cookie set, and confirm that the new URL works in that browser. If it works, then you know your URL and cookie values are good.
  13. In JOSM, create a new TMS image layer and use this new URL, that will have all values hardcoded.
  14. If this works, it will show that image duplicated on every tile. But you know it is properly pulling the high-resolution image with the cookie credentials correctly.
  15. Now change the hardcoded values back to the variables
  16. Here's my final entry: tms[3,15]:https://heatmap-external-b.strava.com/tiles-auth/all/hot/{zoom}/{x}/{y}.png?Key-Pair-Id=MYVALUE1&Policy=MYVALUE2&Signature=MYVALUE3
  • you can only zoom to 15. Anything higher than that and it disappears.

Hopefully this helps someone else. Don't forget, the cookie only lasts a couple weeks I think, so you'll need to get and set the new cookie creds every couple weeks.

comment:23 Changed 5 months ago by solitone

I do get hi-res heatmap data although I don't have a premium account.

comment:24 Changed 5 months ago by drjoshua

I can also see it without premium.
Is it impossible to make it visible above zoomlevel 15 but in a scaled look? I don't mind a little blurry map!

comment:25 Changed 5 months ago by drjoshua

I forgot the first value of '[3,15]' in the URL. Forget my post above!

comment:26 Changed 3 months ago by Don-vip

Component: External imagery sourcePlugin strava_heatmap
Owner: changed from team to allier-osm

comment:27 Changed 2 months ago by daveemtb

Any chance someone could post a dummies guide on how to obtain the Key-Pair-Id, Signature, Policy values please?

Thanks!

comment:28 Changed 2 months ago by solitone

First you need to click on https://www.strava.com/heatmap and log in with your Strava credentials (flag the remember option).
Strava server will send some cookies to your browser, and in one of those cookies you'll find the Key-Pair-Id, Signature, and Policy data.
How to find those data depends on your browser. What are you using?

comment:29 Changed 7 weeks ago by lists@…

Since it is a "closed" oAuth process, and they don't seem willing to let just about anybody in on it, what about trying to get JOSM as an authorised app, and a strava authentication tab in preferences? That way, JOSM could take care of retrieving and storing the oAuth keys, and unlocking the strava imagery choices in the menu...

comment:30 Changed 5 weeks ago by pschonmann

Cc: pschonmann added

comment:31 Changed 5 weeks ago by anonymous

There was a cool tool, called slide, which allowed mappers to automatically correct ways to Strava heat. The developer since then left Strava and couldn't do much about as it's currently broken. Maybe one day it could work again, who knows.

https://github.com/paulmach/slide (check issues)
https://labs.strava.com/slide/

Modify Ticket

Change Properties
Set your email in Preferences
Action
as new The owner will remain allier-osm.
as The resolution will be set.
to The owner will be changed from allier-osm to the specified user.
The owner will change to anonymous
as duplicate The resolution will be set to duplicate.The specified ticket will be cross-referenced with this ticket
The owner will be changed from allier-osm to anonymous.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.