Modify

Opened 15 months ago

Closed 13 months ago

Last modified 13 months ago

#20510 closed defect (fixed)

Default maximum memory setting (-Xmx) for distribution packages

Reported by: wiktorn Owned by: Don-vip
Priority: normal Milestone: 21.04
Component: Core Version:
Keywords: memory jvm performance Cc: stoecker

Description

Currently for our distribution channels we have following maximum heap memory settings:

distribution channel maximum memory
josm.jnlp 2048m
Windows Installer 2048m
macOS package 8192m

Two questions:

  1. Why for macOS we have 4x more memory than for other platforms?
  2. Why do we set this up anyway? As far as I've tested, Java defaults to 25% of physical memory and taking into account current configurations, I'd assume that for 90+% users, this would at least give us the same result or higher. On the other hand, when we have -Xmx occupying high percentage of physical memory, overall performance may be unsatisfying due to high swap usage

For a start, I'd lower the macOS -Xmx to be on par with other platforms.

See: #20443, #20497, #19937

Attachments (0)

Change History (29)

comment:1 Changed 15 months ago by Stereo

The macOS maximum is probably just me scratching my own itch because I was often running into memory limits. The cheapest Mac you can buy in 2021 comes with 8GB, and it's indeed not great if JOSM gobbles it all up.

The "modern" way of doing things is to limit to a percentage: https://support.cloudbees.com/hc/en-us/articles/204859670-Java-Heap-settings-best-practice . As you've seen, the default is 25%. This isn't something we can change at runtime, is it?

comment:2 Changed 15 months ago by wiktorn

No, we can't change that at runtime. Other applications do it, by having separate file that starts JVM, where you can toggle it. But in case of macOS - this file is inside *.dmg and is signed, so no modifications are available..

There is JEP that aims to improve this situation and allow runtime changing of maximum heap size, but probably it's a long way, until it's implemented (if at all).

I vote for let the JRE use it's default 25%.

comment:3 Changed 15 months ago by GerdP

In JOSM a good reason to use a high -Xmx value is when you want to load large files. Another might be when you load lots of images. For normal mapping activities 2G are more than enough. Always presuming you are not using a method with a memory leak too often ;)

comment:4 Changed 15 months ago by Don-vip

Does the JVM really takes 25% by default for JNLP files? I remembered a very low hardcoded value (something like 256Mb)

comment:5 Changed 15 months ago by Don-vip

Keywords: memory jvm performance added

comment:6 Changed 15 months ago by wiktorn

Without testing can't confirm. But as I run jdk1.8.0_171 to launch the jar file, it is setting it to 25% of physical memory. And AFAIR it was not the case earlier with JRE. Hopefully it's the same with JNLP.

If you check this documentation it still mentions, that default 25% should be capped at 1GB, but it's not longer the case.

Oh, and found some SO answer for that. It's since 6u18, that it's always 25% :-) Maybe it's time to hop on ;-)

comment:7 Changed 15 months ago by skyper

Sadly, I do not find a way to use a percentage of physical memory as java option on startup.

comment:8 in reply to:  7 Changed 15 months ago by wiktorn

Replying to skyper:

Sadly, I do not find a way to use a percentage of physical memory as java option on startup.

But can we just move to default by not specifying -Xmx?

comment:9 Changed 15 months ago by wiktorn

Milestone: 21.03

comment:10 Changed 15 months ago by Stereo

Remember it's a maximum, and that most JOSMs will never swallow up all that RAM. I'd much rather have a high maximum that never gets used than the dreaded JOSM out of memory message while I'm in the middle of a complex task I've already invested a lot of time in. If I was limited to 25% of RAM, I'd get that message all the time.

How does -XX:MaxRAMPercentage 75 sound?

comment:11 Changed 15 months ago by skyper

I would go with the default value by not specifying -Xmx. Actually, I use less and do face the "Out of memory" message only after distinct use of validator on many objects.

People, needing more can still specify this in their start script or command.

Last edited 13 months ago by skyper (previous) (diff)

comment:12 in reply to:  10 ; Changed 15 months ago by wiktorn

Replying to Stereo:

Remember it's a maximum, and that most JOSMs will never swallow up all that RAM. I'd much rather have a high maximum that never gets used than the dreaded JOSM out of memory message while I'm in the middle of a complex task I've already invested a lot of time in. If I was limited to 25% of RAM, I'd get that message all the time.

Remember, that we're in GC world and it's up to GC to decide, how much memory JOSM will use. I'd just like to point out that here GerdP noticed, that using -Xmx8G on his machine did cause performance problem, while using -Xmx2G did not. Probably due to heap overflowing to swap.

So setting the -Xmx high invites Java GC to use this memory (though AFAIR, Java is able to reclaim memory back to OS).

And the question is - what is better in case of shortage of memory - OutOfMemoryException or unbearble slowness?

comment:13 Changed 15 months ago by GerdP

Just to make that clear: I used the 8G value only to reproduce the problem. With 64 bit JRE you can set a value that is even higher than the pysical memory, but the massive swapping of pages makes JOSM (and the whole PC) unusable.

I wouldn't set any limit by default and just document the possible JRE options. A JRE that uses almost all available memory is likely to cause more trouble than just some lost OSM changes.

comment:14 Changed 14 months ago by Don-vip

Ticket #10213 has been marked as a duplicate of this ticket.

comment:15 in reply to:  12 Changed 14 months ago by mdk

So setting the -Xmx high invites Java GC to use this memory (though AFAIR, Java is able to reclaim memory back to OS).

You remember right. See details in "4.1 Total Heap" on https://www.oracle.com/java/technologies/tuning-garbage-collection-v50-java-virtual-machine.html#0.0.0.%20Total%20Heap|outline

comment:16 Changed 14 months ago by Don-vip

Milestone: 21.0321.04

comment:17 Changed 14 months ago by simon04

Owner: changed from team to simon04
Status: newassigned

comment:18 Changed 14 months ago by simon04

Resolution: fixed
Status: assignedclosed

In 17711/josm:

fix #20510 - Default maximum memory setting -Xmx for distribution packages

comment:19 Changed 14 months ago by gaben

On the front page, the josm-latest.jnlp file still contains the max-heap-size="2048m" attribute.

Last edited 14 months ago by gaben (previous) (diff)

comment:20 in reply to:  19 ; Changed 13 months ago by Klumbumbus

Resolution: fixed
Status: closedreopened

Replying to gaben:

On the front page, the josm-latest.jnlp file still contains the max-heap-size="2048m" attribute.

The tested jnlp too.

comment:21 in reply to:  20 Changed 13 months ago by skyper

Replying to Klumbumbus:

Replying to gaben:

On the front page, the josm-latest.jnlp file still contains the max-heap-size="2048m" attribute.

The tested jnlp too.

That is ok, as the new tested (milestone 21.4) is not released, yet. But the changes need to be made to both, latest and tested, scripts.

comment:22 Changed 13 months ago by skyper

How are both *.jnlp updated? Last change for both was in January but I did not find a ticket mentioning the files, so far.

comment:23 Changed 13 months ago by simon04

Cc: stoecker added

comment:24 Changed 13 months ago by Don-vip

It's a manual copy. For a long time the only updates to this file were made every two/three years when we upgraded our version of Java.

comment:25 Changed 13 months ago by Don-vip

Owner: changed from simon04 to Don-vip
Status: reopenednew

josm-latest.jnlp updated. I'll update josm.jnlp only when we'll release 21.04. I keep the ticket open to not forget.

comment:26 Changed 13 months ago by simon04

Once r17833 is built, it can be released as milestone:21.04. Afterwards we can update the https://josm.openstreetmap.de/download/josm.jnlp file.

comment:27 Changed 13 months ago by Don-vip

Great! Thanks for taking care of all other stuff. My rocket is launched in just 28 hours :)

comment:28 Changed 13 months ago by Don-vip

Resolution: fixed
Status: newclosed

Done.

comment:29 Changed 13 months ago by simon04

Thank you and all the best for the launch. 🚀

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Don-vip.
as The resolution will be set.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.