IPv6 network trouble in Orange network (FR)

[anonymous]

What steps will reproduce the problem?

1. Use JOSM with a new profile on a computer on IPv6. You can download and upload OSM data.
2. Restart JOSM. It won’t download and upload data anymore.
3. Edit preferences.xml: change <tag key='validated.ipv6' value='true'/> to <tag key='validated.ipv6' value='false'/>. Restart JOSM and it will download and upload data again. But preferences.xml has to be modified at each run.

What is the expected result?

Download and upload data every time by taking into account its parameters.

What happens instead?

Unless you edit preferences.xml at every run, JOSM becomes useless.

Please provide any additional information below. Attach a screenshot if possible.

running C:\Program Files (x86)\JOSM>java -Djava.net.preferIPv6Addresses=true -jar josm-tested.jar as suggested here ​https://wiki.openstreetmap.org/wiki/FR:JOSM didn’t work.

URL:http://josm.openstreetmap.de/svn/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2017-12-31 03:09:43 +0100 (Sun, 31 Dec 2017)
Build-Date:2017-12-31 02:33:46
Revision:13265
Relative:URL: ^/trunk

Identification: JOSM/1.5 (13265 fr) Windows 7 64-Bit
OS Build number: Windows 7 Home Premium (7601)
Memory Usage: 514 MB / 910 MB (237 MB allocated, but free)
Java version: 1.8.0_151-b12, Oracle Corporation, Java HotSpot(TM) 64-Bit Server VM
Screen: \Display0 1920x1200
Maximum Screen Size: 1920x1200
Dataset consistency test: No problems found

Plugins:
+ ElevationProfile (33815)
+ OpeningHoursEditor (33876)
+ PicLayer (33822)
+ RoadSigns (33798)
+ SimplifyArea (33918)
+ apache-commons (33668)
+ buildings_tools (33735)
+ cadastre-fr (33922)
+ download_along (33710)
+ ejml (32680)
+ geotools (33958)
+ jts (32699)
+ measurement (33760)
+ photo_geotagging (33967)
+ poly (33570)
+ reltoolbox (33708)
+ tageditor (33806)
+ turnlanes (33858)
+ turnlanes-tagging (260)
+ turnrestrictions (33780)
+ undelete (33835)
+ utilsplugin2 (33742)
+ waydownloader (33910)
+ waypoint_search (33874)

Last errors/warnings:
- W: No configuration settings found.  Using hardcoded default values for all pools.

[anonymous]

error message

[stoecker]

Do you have some kind of firewall for IPv6 or only a partial IPv6 installation?

JOSM tests IPv6 before setting this parameter, so your system must be able to reach josm.openstreetmap.de via IPv6 and also open a TCP connection.

Please tell more about your IPv6 setup:

• What provider do you use
• Does IPv6 work/work not in your setup.
  • What do you get when calling ip4.stoecker.eu in your browser? What for ip6.stoecker.eu, what for ip.stoecker.eu?
• Whatever else you think may help

[anonymous]

All the tests were done without firewall.
My Internet provider is Orange France. IPv6 works anywhere that I’m aware of.
JOSM used to work fine until 2 months ago more or less. I’ve been a JOSM user for many years on the same computer.

ip4
(IPv4 address)
(Letters-dashes-and-digits).w86-252.abo.wanadoo.fr

ip6.stoecker.eu and ip.stoecker.eu:
2a01:cb08:208:700:3054:3ee3:9f23:61b4
32‐digi_t_he:xade:cima:l_IP:_add:ress
(32‐_digit_hexadecimal_IP_address).ipv6.abo.wanadoo.fr

[Don-vip]

I'm also with Orange France with ipv6 and don't have any problem.
My results are similar:

ip4:
90.89.xxx.xxx
LFbn-xxx.w90-89.abo.wanadoo.fr

ip6:
2a01:cb19:834a:f400:xxxx:xxxx:xxxx:xxxx
(I don't have anything however in "ipv6.abo.wanadoo.fr")

I'm using Fiber access, are you using an ADSL connection?

[Don-vip]

Dirk: I think we should also test access to the OSM API in addition to JOSM website in the ipv6 check.

[stoecker]

Replying to Don-vip:

Dirk: I think we should also test access to the OSM API in addition to JOSM website in the ipv6 check.

No. This first test is already an ugly thing. Another will not improve situation.

@anonym:

All tests say you have a working IPv6 connection.

What happens when you connect with these links (api.openstreetmap.org)?:

• ​https://[2001:630:12:500:219:bbff:fe39:3d9e]/
• ​https://[2001:630:12:500:21a:4bff:fea5:fd2a]/
• ​https://[2001:630:12:500:219:bbff:fe39:8aba]/

You should get a security warning (don't accept the certificate, except temporary if you want, you will then reach the normal OSM webpage).

FIX: You can disable IPv6 completely when you set "prefer.ipv6" to "false" in the expert settings (default is "auto").

Something is special at your system, and I doubt we can do much to fix it. If you want to help finding the reason I nevertheless would be interested in knowing the cause for the troubles, but that means you need to install some network tracing software like wireshark and do some tests.

[Don-vip]

@anonym: another test, can you please go to ​https://ip.lafibre.info/ and tell us what TCP ports are being used, and if they're displayed in green or red?

[Don-vip]

I have several reports of French users struggling with this issue. It looks like Orange changed something on their ADSL network infrastructure recently and did something wrong...

[anonymous]

Hi.

So, on ​https://ip.lafibre.info/ everything is green. Ports are:

Port TCP source utilisé par votre connexion IPv4 : TCP 58351
Port TCP source utilisé par votre connexion IPv6 : TCP 58350 (Plus d'informations)
Port TCP destination utilisé : TCP 443

As for api.openstreetmap.org, nslookup finds the same addresses as you + the IPv4 addresses, but Firefox doesn’t connect to your IPv6 links.

Your workaround on prefer.ipv6 to false works and JOSM connects now every time, probably in IPv4.

Update: while trying to track the outgoing network requests, I set the maximum alerts on the Comodo firewall (which was NOT running at the time the tests on JOSM were done) and discovered that it has an advanced option to filter IPv6, which is unticked by default. I activated the check box and discovered that the JOSM problem disappeared!

So, it’s either a bug on Comodo firewall or a conflict between Comodo and Java. For now I let the IPv6 filter activated on the Comodo firewall. I don’t know why it is unchecked by default and what the effect of activating it is.
Thanks for your help and I consider this case closed, though I still don’t understand why I can run JOSM once with no problems, but not the subsequent times.

[stoecker]

Replying to Don-vip:

I have several reports of French users struggling with this issue. It looks like Orange changed something on their ADSL network infrastructure recently and did something wrong...

Id like to get a TCP connection log report either using tcpdump, wireshark or any other PCAP logger which is compatible with wireshark.

As the initial connect comes through, but not normal connections maybe the MTU and fragmentation is handled wrong.

[stoecker]

Replying to anonym:

Update: while trying to track the outgoing network requests, I set the maximum alerts on the Comodo firewall (which was NOT running at the time the tests on JOSM were done) and discovered that it has an advanced option to filter IPv6, which is unticked by default. I activated the check box and discovered that the JOSM problem disappeared!

Depending on what the filter does maybe you completely disable IPv6 as well → JOSM will fallback to IPv4 then. Hard to say without logs of what's going on ;-)

So, it’s either a bug on Comodo firewall or a conflict between Comodo and Java. For now I let the IPv6 filter activated on the Comodo firewall. I don’t know why it is unchecked by default and what the effect of activating it is.

Very likely your IPv6 connection is broken in some way, but in most applications like web browsers you wont notice this as they have fallbacks and try all IP addresses. Java/JOSM is a bit simpler and only uses the first valid IP address is has.

Thanks for your help and I consider this case closed, though I still don’t understand why I can run JOSM once with no problems, but not the subsequent times.

That's easy. As Java IPv6 support is suboptimal we must know whether to set IPv6 in early JOSM startup. Thus we test IPv6 and activate it later. So first connect is always IPv4 as long as IPv4 exists.

The case that we can connect to IPv6 hosts fine, but later on stuff fails was not expected. For such cases only "Happy Eyeballs" or similar algorithms work, but that is a much more complicated topic.

[Don-vip]

Replying to stoecker:

Id like to get a TCP connection log report either using tcpdump, wireshark or any other PCAP logger which is compatible with wireshark.

I just sent an e-mail on @talk-fr to ask for volunteers to come here to provide this.

[anonymous]

Now my JOSM doesn’t save the tag <tag key='validated.ipv6' value='*'/> anymore, but it still works.

To answer your question, Comodo firewall shows me that it connects to 2a01:4f8:c17:2906::2] - ICMPv6, but not to other protocols or IPv6 addresses. Sometimes to some IPv4 address.

When I have time, I will run JOSM on Linux to see how it runs.

[francois.lacombe]

IPv6 connectivity issue confirmed on Orange side.
HTTP works normally on other carriers.

Orange routing experts has been advised minutes ago
​https://twitter.com/InfosReseaux/status/949052812033478656

[Don-vip]

Dirk,

Someone posted a quite interesting message on talk-fr:

IPv6 to ADSL at Orange does not use native IPv6 routing (unlike fiber) but an encapsulation of an IPv6 tunnel on IPv4 and it seems that this limits the size of the "MTU" in TCP as well as in UDP or raw IP (ICMP/ping) and that this also poses a problem for HTTPS.
IPv6 over ADSL at Orange has no interest, the IPv6 tunnel provider at Orange is very poorly performing (and overloaded on its servers not suitable for broadband usage by many users). It can be deactivated in the Orange box settings (although we know that it will be difficult to connect to Asian mobiles that are only reachable in IPv6, possibly through a tunnel).
But I don't see any problems on the Orange fiber (and I don't see any problems on the Asian IPv6 mobiles...) and I don't see any problems even to watch a Youtube video in native IPv6 (it's possible that Google itself can provide the IPv6 to IPv4 tunnel if necessary or that it can detect MTU problems too short for some operators).
Orange has never been clear about its IPv6 deployment which is very late in the market, and not supported correctly on a number of its obsolete firmware ADSL boxes or on a number of its DSLAMs (or regional traffic collectors) not up to date to make native IPv6 tunnelless via IPv4 (or because there are compatibility issues with other operators that Orange ensures.
You can detect MTU problems with some tunnel providers, and other IPv6 conenctivity defects with ICMP-based tools, or on several IPv6 test sites (including ​​http://ipv6-test.com/ which tells you if you really have a native IPv6 connection).

I confirm I have no problem with IPv6 with my Orange Fiber access.

[Don-vip]

Is it possible from Java to detect if we're using native IPv6 or IPv6 over IPv4?

[francois.lacombe]

I confirm I have no problem with IPv6 with my Orange Fiber access.

Issues occur on fibre accesses too.
IPv6 stack isn't different between xDSL or FTTH accesses.

Message posted on talk-fr regarding issues in xDSL different from fibre isn't right.
Investigations pending by Orange staff

[Don-vip]

OK thank you François. Waiting for your feedback :)

[stoecker]

Replying to Don-vip:

Is it possible from Java to detect if we're using native IPv6 or IPv6 over IPv4?

That largely depends on the used method and the resulting IP you get. Whatever Orange does the IPv6 addresses given above are native, so no chance to detect it except with a "Orange" workaround by blocking a certain network segment - I don't like that at all.

Should we add a startup note to JOSM telling about this issue and a recommendation to contact Orange in case of trouble? Only way to solve IPv6 issues is to increase pressure on the providers to fix their systems. IPv6 is much better than IPv4, but providers must know what they do and not setup some half-implemented solutions.

Now with 20% IPv6 and exponential increase it is highest time to get the issues solved for providers: ​https://www.google.de/ipv6/statistics.html Even JOSM with its "In doubt use IPv4" policy has around 10% IPv6 now.

With Hetzner YEARS AGO I permanently contacted them for each IPv6 issue until they finally reached the same quality as for IPv4, but that was in the 0.x% IPv6 traffic area.

talk-fr thread for reference: ​https://lists.openstreetmap.org/pipermail/talk-fr/2018-January/087298.html

[anonymous]

Replying to stoecker:

That largely depends on the used method and the resulting IP you get. Whatever Orange does the IPv6 addresses given above are native, so no chance to detect it except with a "Orange" workaround by blocking a certain network segment - I don't like that at all.

Sounds like a hardware problem on a particular router in Marseille.
Some subscribers including Don-Vip manage to reach those IPv6 addresses on a Orange fibre connections because they are not going through this node.
No filtering and no specific DPI crap involved.
This is currently under investigation by network engineers.

Should we add a startup note to JOSM telling about this issue and a recommendation to contact Orange in case of trouble? Only way to solve IPv6 issues is to increase pressure on the providers to fix their systems. IPv6 is much better than IPv4, but providers must know what they do and not setup some half-implemented solutions.

Orange sounds like the only provider in France to implement native IPv6 networking while others despite rolling it out earlier only provide 6to4 or 6rd.
The problem isn't related to build but to run operations.
Please don't add anything specific to JOSM regarding this.

Stay tuned.
(francois.lacombe, not logged in)

[stoecker]

Replying to anonym:

The problem isn't related to build but to run operations.
Please don't add anything specific to JOSM regarding this.

Before I really consider a workaround much more must happen ;-)

[stoecker]

P.S. If/When they fix it, a little "Thanks" is like positive "pressure"...

[Don-vip]

Ticket #15735 has been marked as a duplicate of this ticket.

[anonymous]

Being the original poster, I pop saying that my connection is from Orange France on optical fibre.

[francois.lacombe]

Could you please send me (or post here) some traceroute output from your location to google or whatever please ?

[anonymous]

Here is my tracert to api.openstreetmap.org

1    <1 ms    <1 ms    <1 ms  livebox.home [~myIPv6~]
2     3 ms     2 ms     1 ms  ~myIPv6~.ipv6.abo.wanadoo.fr [~myIPv6~]
3     2 ms     2 ms     2 ms  2a01:cfc4:0:500::b
4     7 ms     7 ms     7 ms  bundle-ether104.auvtr4.aubervilliers.opentransit.net [2a01:cfc4:0:500::7]
5     2 ms     6 ms     3 ms  et5-0-3-0.pastr3.Pastourelle.opentransit.net [2001:688:0:2:4::12]
6     8 ms     9 ms    10 ms  2001:2000:3080:cd::1
7     *        *        *     Délai d'attente de la demande dépassé.

[anonymous]

I uninstalled Comodo as I was tired of it recently and… now I have the same problem as mentioned before. I’m puzzled.
preferences.xml has <tag key='validated.ipv6' value='true'/> again
Maybe it wasn’t a problem from Comodo firewall after all.

[Don-vip]

@anonymous OP: according to ​https://lists.openstreetmap.org/pipermail/talk-fr/2018-January/087440.html the problem may be fixed, can you please test again?

[francois.lacombe]

This is related to IPv6 London Gateway fixed last night.
We're still waiting for Marseille gateway.

Don't forget to provide mtr/traceroute if you're still experiencing issues.

[anonymous]

It seems to be working to me. Thanks for the great news and I will keep you updated if anything goes wrong.

[stoecker]

Hmm, actually otherhardware ;-)