OpenWebStart/Java 17 migration

[Don-vip]

Things are starting to take shape with what comes after Java WebStart (see #16047):

​https://openwebstart.com/

Original 2019 plan:

On the current roadmap, the first version will be released end of October. A macro planning for a Java 8 => Java 11 transitions for all JOSM users would roughly look like this:

• September 2019: we start testing OpenWebStart on all platforms. Likely we'll found a lot of bugs
• November 2019 : first OpenWebStart version. Unlikely to fix all bugs we'll find
• Somewhere in 2020: OpenWebStart version without any bug impacting us, we start asking everyone to switch
• End of 2020: End of Java WebStart support by Oracle for Java 8. We force everyone to switch
• Somewhere in 2021: Enough JOSM users have switched to OpenWebStart so we can consider moving the codebase to Java 11.

New 2021 plan:

• 2021-03-28: ask Oracle Java WebStart users to switch to OpenWebStart => r17679
• 2021-08-22: new Windows package that includes Java 16 => r18151:18155
• 2021-08-22: include JavaFX 16 in macOS and Windows packages => r18161
• 2021-08-22: update JNLP files to request Azul JVM from OpenWebStart as it includes JavaFX => r18158:18159
• 2021-08-22: update Debian/Ubuntu launch script to depend on openjfx => r18160
• September 2021: Java 17 is released. Switch macOS / Windows packages to Java 17 and JavaFX 17, update Debian/Ubuntu launch script to prefer 17 over 11 and 8
• April 2022: Ubuntu 22.04 LTS is released and ships Java 17
• somewhere in 2022: Enough JOSM users are now using Java 17+ so we can consider moving the codebase to Java 17.

[Don-vip]

First problem: #17632

[Don-vip]

First alpha version (0.2.0) is available for download: ​https://openwebstart.com/download/

[mdk]

Version 1.1.1 is released (at 16.12.2019)

[Don-vip]

Second problem: #18390 => ​https://github.com/AdoptOpenJDK/IcedTea-Web/pull/564 => ​https://github.com/AdoptOpenJDK/IcedTea-Web/releases/tag/icedtea-web-2.0.0-alpha9 => ​https://github.com/karakun/OpenWebStart/releases/tag/v1.1.4

[Don-vip]

AdoptOpenJDK API now available through https://josm.openstreetmap.de/remote/adoptopenjdk-api/ (#18723)

[Don-vip]

​https://wiki.ubuntu.com/BionicBeaver/ReleaseNotes/

As of 18.04.4, OpenJDK 11 is the default in 18.04.

OpenJDK 8 has moved to universe and will remain available there for the life of 18.04, to provide migration time for packages, custom applications, or scripts that can't be build with OpenJDK 11. OpenJDK 8 will be updated in 18.04 until Ubuntu 16.04 LTS reaches EOL in April 2021.

[Don-vip]

Another issue: #18737 => ​https://github.com/AdoptOpenJDK/IcedTea-Web/issues/595

[Don-vip]

AdoptOpenJDK officially refused to ship OpenJFX in their binary distributions:
​https://github.com/AdoptOpenJDK/TSC/issues/27#issuecomment-607663428

So the best distribution for us would be ​Azul Zulu and ​Bellsoft's Liberica.

[taylor.smock]

Another issue: #19044 (Azul Zulu 11.0.6/AdoptOpenJDK 11.0.6). It looks like jdk.swing.interop.SwingInterOpUtils doesn't exist, for whatever reason. Oddly enough, it is never directly called by us (its called by JFXPanel, which is in the java distribution). I haven't filed a bug upstream, since I haven't run with a "standard" version of Azul Zulu yet.

EDIT: Not reproducible with Azul Zulu or AdoptOpenJDK from CLI.

Possibly related to ​https://github.com/AdoptOpenJDK/IcedTea-Web/issues/595 .

[mdk]

It looks like ​​https://github.com/AdoptOpenJDK/IcedTea-Web/issues/595 is fixed in the actual release 1.1.7

[jBeata]

Do you have a more clear deadline for moving the code-base to Java11? It will happen in the first half of 2021 or the second?

[stoecker]

Replying to jBeata:

Do you have a more clear deadline for moving the code-base to Java11? It will happen in the first half of 2021 or the second?

Usually when less than 5% of our users use older versions. Currently 70% of our users use Java 8. About 20% use Java 11.

[Don-vip]

Oracle changed again their plans concerning Java 8. It was previously expected they stop releasing public updates of Java 8 on java.com (the version used by nearly all our Windows users) at the end of 2020.

Now ​the plan is:

Oracle will continue to provide free public updates and auto updates of Java SE 8 indefinitely for Personal, Development and other Users via java.com. Oracle will provide at least 18 months notice on this page and other communication channels if an end of availability date is set. [...]
Oracle does not plan to migrate desktops from Java SE 8 to later versions via the auto update feature. This includes the Java Plugin and Java Web Start. [...]

So we won't see any major decrease of Java 8 usage until we force this change by promoting OpenWebStart and Java 11 to our users. I didn't start to work on this yet.

Given on our past experience, it takes months between the time we start to deprecate a version of Java and the time enough people abandoned it, so I wouldn't see a Java 11 migration before Q2 2021 at best.

[stoecker]

Replying to Don-vip:

Q2 2021 at best.

Still very optimistic. Q4 2021 or later seems more realistic ;-) Even Linux has still about 30% Java 8.

P.S. Added --os option to our checkjosm tool :-)

[Don-vip]

I meant S2, with Q4 in mind actually :)

[jBeata]

Thanks for the clarification. We also use Java 8 since the minimum version for running JOSM is Java 8. But we plan to switch as soon as JOSM main version is updated to 11.

[Don-vip]

Current numbers:

Java Main Version --> 8 (7174, 67.6%) 9 (21,  0.2%) 10 (24,  0.2%) 11 (2130, 20.1%) 12 (167,  1.6%) 13 (248,  2.3%) 14 (299,  2.8%) 15 (532,  5.0%) 16 (10,  0.1%) 17 (7,  0.1%)

I'm really happy to see that OpenWebStart really kicks in: several versions released, 5 companies seem to sponsor Karakun on front page, nearly 200 developers have starred the project on GitHub. Software is now available on Windows, mac and Linux. It's a complete and viable replacement to Oracle WebStart.

I'll see how to detect Oracle WebStart and suggest users to switch to OpenWebStart. At the same time they do that, they'll switch from Oracle JRE 8 to Azul JRE 11.

@Dirk would it be possible to update checkjosm to get vendor stats? I'd like to see the percentage of users running an Oracle runtime versus those who do not.

[stoecker]

That information isn't available in the User-Agent, so I can't display it :-)

[Don-vip]

Replying to stoecker:

That information isn't available in the User-Agent, so I can't display it :-)

Ah, I thought it was sent. I'll check only Java 8 then. Should be the same thing.

[Don-vip]

I just fixed #18737 which was the only blocker to OpenWebStart migration. Let's get rid of Oracle Java WebStart.

[Don-vip]

Vote for your preferred rocket icon! Which one looks the best? 1, 2, 3, 4?

[simon04]

I vote for 4️⃣ :-)

[simon04]

Awesome, successfully tested (ticket #18737) on macOS! On Windows and macOS the status report lacks any indication that JOSM is running via webstart:

Relative:URL: ^/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2021-03-26 22:41:31 +0100 (Fri, 26 Mar 2021)
Revision:17674
Build-Date:2021-03-27 02:30:56
URL:https://josm.openstreetmap.de/svn/trunk

Identification: JOSM/1.5 (17674 en) Mac OS X 10.16
OS Build number: macOS 11.2.3 (20D91)
Memory Usage: 410 MB / 2048 MB (171 MB allocated, but free)
Java version: 16+36, Azul Systems, Inc., OpenJDK 64-Bit Server VM
Look and Feel: com.formdev.flatlaf.FlatLightLaf
Screen: Display 1 1440×900 (scaling 2,00×2,00) Display 2 3008×1692 (scaling 2,00×2,00)
Maximum Screen Size: 3008×1692
Best cursor sizes: 16×16→16×16, 32×32→32×32
VM arguments: [--add-modules=java.scripting,java.sql, --add-exports=java.desktop/com.apple.eawt=ALL-UNNAMED, --add-exports=java.desktop/com.sun.imageio.spi=ALL-UNNAMED, --add-exports=java.desktop/com.sun.imageio.plugins.jpeg=ALL-UNNAMED, --add-exports=javafx.graphics/com.sun.javafx.application=ALL-UNNAMED, --add-exports=jdk.deploy/com.sun.deploy.config=ALL-UNNAMED, --add-opens=java.base/java.lang=ALL-UNNAMED, --add-opens=java.base/java.nio=ALL-UNNAMED, --add-opens=java.base/jdk.internal.loader=ALL-UNNAMED, --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED, --add-opens=java.desktop/javax.imageio.spi=ALL-UNNAMED, --add-opens=java.desktop/javax.swing.text.html=ALL-UNNAMED, --add-opens=java.prefs/java.util.prefs=ALL-UNNAMED, -Djava.util.Arrays.useLegacyMergeSort=true, --add-exports=java.base/sun.net.www.protocol.jar=ALL-UNNAMED,java.desktop, --add-exports=java.base/jdk.internal.util.jar=ALL-UNNAMED,java.desktop, --add-exports=java.base/com.sun.net.ssl.internal.ssl=ALL-UNNAMED,java.desktop, --add-reads=java.naming=ALL-UNNAMED,java.desktop, --add-exports=java.desktop/sun.awt.X11=ALL-UNNAMED,java.desktop, --add-exports=java.desktop/sun.applet=ALL-UNNAMED,java.desktop,jdk.jsobject, --add-exports=java.base/sun.security.action=ALL-UNNAMED,java.desktop, --add-reads=java.base=ALL-UNNAMED,java.desktop, --add-exports=java.base/sun.net.www.protocol.http=ALL-UNNAMED,java.desktop, --add-exports=java.naming/com.sun.jndi.toolkit.url=ALL-UNNAMED,java.desktop, --add-exports=java.base/sun.security.util=ALL-UNNAMED,java.desktop,ALL-UNNAMED, --add-reads=java.desktop=ALL-UNNAMED,java.naming, --add-exports=java.desktop/sun.awt=ALL-UNNAMED,java.desktop, --add-exports=java.base/sun.security.x509=ALL-UNNAMED,java.desktop,ALL-UNNAMED, --add-exports=java.desktop/javax.jnlp=ALL-UNNAMED,java.desktop, --add-exports=java.base/sun.security.provider=ALL-UNNAMED,java.desktop, --add-exports=java.base/sun.security.validator=ALL-UNNAMED,java.desktop]

Plugins:
+ ImportImagePlugin (35567)
+ apache-commons (35524)
+ ejml (35458)
+ flatlaf (35703)
+ geotools (35458)
+ jts (35458)
+ log4j (35458)

This statement might help us to identify OpenWebStart:

Class.forName("com.openwebstart.launcher.OpenWebStartLauncher")

[skyper]

Replying to Don-vip:

Vote for your preferred rocket icon! Which one looks the best? 1, 2, 3, 4?

I like 1 or 4. Maybe, the dark brown on the outside wings could be a bit lighter. With the dark mode icons need to respect dark and light background and often contrasting outlines like 1 and 2 show are helpful.

[simon04]

When starting, OpenWebStart shows this information message:

App name on macos screen menu is Boot · Issue 325 · karakun/OpenWebStart · ​https://github.com/karakun/OpenWebStart/issues/325

[Don-vip]

Replying to simon04:

This statement might help us to identify OpenWebStart:

Class.forName("com.openwebstart.launcher.OpenWebStartLauncher")

I plan to use this (not tested yet):

    /**
     * Determines whether JOSM has been started via Web Start (JNLP).
     * @return true if JOSM has been started via Web Start (JNLP)
     * @since xxx
     */
    public static boolean isRunningWebStart() {
        try {
            // See http://stackoverflow.com/a/16200769/2257172
            return Class.forName("javax.jnlp.ServiceManager") != null;
        } catch (ClassNotFoundException e) {
            return false;
        }
    }

    /**
     * Determines whether JOSM has been started via Oracle Java Web Start.
     * @return true if JOSM has been started via Oracle Java Web Start
     * @since 15740
     */
    public static boolean isRunningJavaWebStart() {
        return isRunningWebStart() && Package.getPackage("com.sun.javaws") != null;
    }

    /**
     * Determines whether JOSM has been started via Open Web Start (IcedTea-Web).
     * @return true if JOSM has been started via Open Web Start (IcedTea-Web)
     * @since xxx
     */
    public static boolean isRunningOpenWebStart() {
        return isRunningWebStart() && Package.getPackage("net.adoptopenjdk.icedteaweb") != null;
    }

[Don-vip]

This is the upgrade text I came with. Comments?

For the icon, I've started a ​poll on Twitter as well, I hope a clear winner comes out :D

[Don-vip]

Replying to Don-vip:

    public static boolean isRunningOpenWebStart() {
        return isRunningWebStart() && Package.getPackage("net.adoptopenjdk.icedteaweb") != null;
    }

And we should also be aware of upcoming changes that will probably affect package names for ITW ​https://blog.adoptium.net/2021/03/eclipse-adoptium-announcement/

[simon04]

Replying to Don-vip:

This is the upgrade text I came with. Comments?

Two small suggestions:

-Oracle implementation (2x)
+an Oracle implementation

-as a new product: OpenWebStart.
+as a new product: OpenWebStart

What is the difference between "OK" and "Download OpenWebStart"?

You you plan to add this dialog for the 21.03 release (I'm referring to i18n)?

[Don-vip]

Replying to simon04:

Two small suggestions:

-Oracle implementation (2x)
+an Oracle implementation

-as a new product: OpenWebStart.
+as a new product: OpenWebStart

Thanks.

What is the difference between "OK" and "Download OpenWebStart"?

OK does nothing, it just closes the dialog. It's the same behaviour as current MainApplication.askUpdateJava dialog.

You you plan to add this dialog for the 21.03 release (I'm referring to i18n)?

Yes, tomorrow evening.

[stoecker]

Replying to Don-vip:

This is the upgrade text I came with. Comments?

Remove the remaining sentence after "Java 11" and join the two sentences with "but".

I'd vote for icon 4 (icon 1 is second place).

[Don-vip]

New version taking into account Simon and Dirk feedback:

[simon04]

Looks good! 👍🏿

[stoecker]

Actually I meant to join the first two sentences, but this is even better ;-)

[Don-vip]

Good :) Thank you :)

[Don-vip]

In 17679/josm:

see #17858 - ask Oracle Java WebStart users to switch to OpenWebStart

Rocket icon from ​https://github.com/twitter/twemoji/blob/v13.0.2/assets/svg/1f680.svg

[simon04]

Java Warnings (Package.getPackage is deprecated since Java 9):

/Users/simon/src/josm/src/org/openstreetmap/josm/tools/Utils.java:1742: warning: [deprecation] getPackage(String) in Package has been deprecated
        return isRunningWebStart() && Package.getPackage("com.sun.javaws") != null;
                                             ^
/Users/simon/src/josm/src/org/openstreetmap/josm/tools/Utils.java:1751: warning: [deprecation] getPackage(String) in Package has been deprecated
        return isRunningWebStart() && Package.getPackage("net.adoptopenjdk.icedteaweb") != null;
                                             ^

[Don-vip]

In 17692/josm:

see #17858 - fix deprecation warnings

[simon04]

Replying to simon04:

When starting, OpenWebStart shows this information message:

"The proxy 'System proxy' does not support 'Passive FTP Mode (PASV)'"

Only affects macOS, see ​https://github.com/karakun/OpenWebStart/blob/26c8990af7f2214b290d1056cd27a7f78f4477a6/openwebstart/src/main/java/com/openwebstart/proxy/mac/MacProxyProvider.java#L36-L38

Fixed by selecting "No Proxy" in the OpenWebStart Settings:

[gaben]

On Windows, the WebStart dialogue pops up and not steals the focus, sometimes resulting in a JOSM 'loading' indefinitely. The same applies to Java update notification windows. The only way to resolve is to click somewhere on the splash screen.

URL:https://josm.openstreetmap.de/svn/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2021-04-01 23:17:01 +0200 (Thu, 01 Apr 2021)
Build-Date:2021-04-01 21:46:03
Revision:17702
Relative:URL: ^/trunk

Identification: JOSM/1.5 (17702 hu) Windows 10 64-Bit
OS Build number: Windows 10 Pro for Workstations 2009 (19042)
Memory Usage: 912 MB / 1820 MB (719 MB allocated, but free)
Java version: 1.8.0_281-b09, Oracle Corporation, Java HotSpot(TM) 64-Bit Server VM
Look and Feel: com.sun.java.swing.plaf.windows.WindowsLookAndFeel
Screen: \Display0 1920×1200 (scaling 1,00×1,00)
Maximum Screen Size: 1920×1200
Best cursor sizes: 16×16→32×32, 32×32→32×32
System property file.encoding: Cp1250
System property sun.jnu.encoding: Cp1250

[nkamapper]

I have an old iMac running High Sierra, which is not supported by OpenWebStart. The iMac cannot be upgraded to a newer OS. Is there a way to get an earlier version of josn.jnlp, which would work?

[skyper]

Not on the official side, I guess, but note, *.jnlp are text files which can be modified and created with any text editor.

Another solution is to use .jar files directly. Starting should work from file manager or with java -jar path/filename from the console or with a simple start script including the java command. See Download.

[Don-vip]

We will only support OpenWebStart for JNLP. However in your case you can use the macOS build directly instead of WebStart. There is also the brew cask package on macOS.

[taylor.smock]

Somewhere in 2021: Enough JOSM users have switched to OpenWebStart so we can consider moving the codebase to Java 11.

Replying to Don-vip:

Current numbers:

Java Main Version --> 8 (7174, 67.6%) 9 (21,  0.2%) 10 (24,  0.2%) 11 (2130, 20.1%) 12 (167,  1.6%) 13 (248,  2.3%) 14 (299,  2.8%) 15 (532,  5.0%) 16 (10,  0.1%) 17 (7,  0.1%)

Have these numbers changed significantly? As in, will we be able to move to Java 11 somewhere in 2021, or is it going to be somewhere in 2022?

[stoecker]

Looks still similar (I stripped the absolute values, only left the %):

Java Main Version --> 8 (66.4%) 9 (0.1%) 10 (0.2%) 11 (21.1%) 12 (1.0%) 13 (1.7%) 14 (2.3%) 15 (0.9%) 16 (5.6%) 17 (0.5%) 18 (0.3%)

Values of 15 now moved to 16, but the others are mainly unmoved.

[taylor.smock]

OK. We probably won't be moving to Java 11 this year then.
Thanks for looking.

[stoecker]

It seems the majority of newer Java versions comes from newer Linux systems.

[Don-vip]

Good news: most of Linux and mac users now use Java >= 11.
Bad news: almost 90% of Windows users (Windows users being about two thirds of the JOSM user base) are still using Java 8.

But it's not surprising as Oracle JRE will never prompt Java 8 users to update. And apart of the Java WebStart > OpenWebstart migration message, I still haven't make anything to ask "regular" JRE-based users to transition from an Oracle JRE to something else, so now would be a good time.

From ​https://rafael.codes/openjdk/ I think we should promote these two OpenJDK distributions:

• Liberica JDK from Bell: ​https://bell-sw.com/
• Zulu from Azul: ​https://www.azul.com/downloads/?package=jdk

They both ship OpenJFX, which could be handy for future long-term JOSM developments.

[taylor.smock]

Replying to Don-vip:

Bad news: almost 90% of Windows users (Windows users being about two thirds of the JOSM user base) are still using Java 8.

I think I remember something somewhere talking about Minecraft and Java 16 being the baseline. Hopefully that pushes most of the users on Windows to Java 16 (and 16 > 11 :) ).

Do we want to use jpackage for the Windows installer as well (we are already using it for Mac)? This may also help us push to Java 11+. But it does increase size (15 mb -> 38mb).

They both ship OpenJFX, which could be handy for future long-term JOSM developments.

Good to know -- Mapillary was asking me (again) if I could reuse their Javascript based image viewer, which requires either (1) a JS engine that supports newer features (no native Java ones) or (b) embedding a web browser and distributing that for all the platforms JOSM supports.

[stoecker]

Replying to taylor.smock:

Good to know -- Mapillary was asking me (again) if I could reuse their Javascript based image viewer, which requires either (1) a JS engine that supports newer features (no native Java ones) or (b) embedding a web browser and distributing that for all the platforms JOSM supports.

We did that in the past with a tool called webkit-image, which rendered a URL and saved the resulting image. This was necessary to follow the Yahoo guidelines for their service. Was an ugly workaround, but it worked. Don't know if JOSM still has the code for this or if it was removed.

[Don-vip]

Replying to stoecker:

Don't know if JOSM still has the code for this or if it was removed.

Removed long ago ;) r11578:11581

[Don-vip]

Replying to taylor.smock:

Do we want to use jpackage for the Windows installer as well (we are already using it for Mac)? This may also help us push to Java 11+. But it does increase size (15 mb -> 38mb).

Yes, can be an option too.

[gaben]

I still use Java 8 on Windows because in later versions they changed how scaling works and it is unusable for me. On Linux there is no difference :)
Will try again and report back if something improved in the meantime, which potentially means minus one Java 8 user.

[taylor.smock]

Replying to Don-vip:

Replying to stoecker:

Don't know if JOSM still has the code for this or if it was removed.

Removed long ago ;) r11578:11581

author of removal: stoecker :)

And yes, it looks like an ugly workaround. AKA not something I want to do. I kind of want to try to avoid workarounds, if at all possible. They tend to break.

But if JavaFX is an option in the future, I believe MS Streetside has a 360 image viewer, which would probably fix the problem anyway (ignoring the perennial "2048px is too little for 360" comments).

[taylor.smock]

Replying to gaben:

I still use Java 8 on Windows because in later versions they changed how scaling works and it is unusable for me. On Linux there is no difference :)
Will try again and report back if something improved in the meantime, which potentially means minus one Java 8 user.

Stupid question: Have you reported a bug to the JDK maintainers? Or is the scaling issue intentional?

[stoecker]

Replying to taylor.smock:

Don't know if JOSM still has the code for this or if it was removed.

Removed long ago ;) r11578:11581

author of removal: stoecker :)

Ah no. That's unfair. It was broken before.

[anonymous]

Replying to taylor.smock:

Stupid question: Have you reported a bug to the JDK maintainers? Or is the scaling issue intentional?

It was reported years ago and as far as I know it's an intentional fix for another scaling issue.
If you are interested I can collect the JDK ticket(s) and SO questions and a possible solution if there is any (should be).

Edit: the login cookie expired, it was me (gaben).

[Don-vip]

Ah great, we can give an hint to OpenWebStart about the vendor we prefer:
​https://openwebstart.com/docs/OWSGuide.html#_specify_a_specific_vendor_in_the_jnlp_file

So if we set "Azul" or "BellSoft", OpenWebStart will download this JVM instead of AdoptOpenJDK (which doesn't include JavaFX).

It seems we can only specify a single vendor, so which one should we use between the two? Both seem OK.

[Don-vip]

In 18158/josm:

see #17858 - add custom WebStart DTD with OpenWebStart vendor extension, as it is not (yet?) provided

• ​https://openwebstart.com/docs/OWSGuide.html#_specify_a_specific_vendor_in_the_jnlp_file
• ​https://stackoverflow.com/a/62633477/2257172
• ​http://java.sun.com/dtd/JNLP-6.0.dtd

[Don-vip]

In 18159/josm:

see #17858 - request Azul JVM from OpenWebStart and add following JavaFX modules: controls,media,swing,web

[Don-vip]

In 18160/josm:

see #17858 - make Debian/Ubuntu package depend on openjfx and add following JavaFX modules: controls,media,swing,web

[Don-vip]

In 18161/josm:

see #17083, see #17858 - include JavaFX 16 in macOS/Windows packages

[Don-vip]

I think we have now a good chance to switch to Java 17 somewhere in 2022. This migration will have been absolutely horrible, the worst in my Java development experience.