Attribute display involving CJK crashes JOSM

[jengelh@…]

What steps will reproduce the problem?

1. Download object 35578845 for example.
2. Select it.

What is the expected result?
That the key-value pairs are shown in the "Tags / Memberships" window.

What happens instead?
JOSM throws an exception if the "Tags / Memberships" window is enabled.

Please provide any additional information below. Attach a screenshot if
possible.

Repository Root: http://josm.openstreetmap.de/svn
Build-Date: 2014-01-29 15:23:51
Last Changed Author: Don-vip
Revision: 6767
Repository UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
URL: http://josm.openstreetmap.de/svn/trunk
Last Changed Date: 2014-01-29 16:18:16 +0100 (Wed, 29 Jan 2014)
Last Changed Rev: 6767

Identification: JOSM/1.5 (6767 en) Linux openSUSE 13.1 (Bottle) (x86_64)
Memory Usage: 173 MB / 910 MB (94 MB allocated, but free)
Java version: 1.7.0_51, Oracle Corporation, OpenJDK 64-Bit Server VM
VM arguments: [-Xms64M, -Xmx1024M, -XX:+UseParallelGC, -XX:+UseAdaptiveSizePolicy]
Dataset consistency test: No problems found


java.lang.ArrayIndexOutOfBoundsException: -38795802
	at sun.font.FileFontStrike.getCachedGlyphPtr(FileFontStrike.java:472)
	at sun.font.FileFontStrike.getSlot0GlyphImagePtrs(FileFontStrike.java:438)
	at sun.font.CompositeStrike.getGlyphImagePtrs(CompositeStrike.java:115)
	at sun.font.GlyphList.mapChars(GlyphList.java:272)
	at sun.font.GlyphList.setFromString(GlyphList.java:244)
	at sun.java2d.pipe.GlyphListPipe.drawString(GlyphListPipe.java:71)
	at sun.java2d.pipe.ValidatePipe.drawString(ValidatePipe.java:165)
	at sun.java2d.SunGraphics2D.drawString(SunGraphics2D.java:2867)
	at sun.swing.SwingUtilities2.drawString(SwingUtilities2.java:552)
	at sun.swing.SwingUtilities2.drawStringUnderlineCharAt(SwingUtilities2.java:584)
	at javax.swing.plaf.synth.SynthGraphicsUtils.paintText(SynthGraphicsUtils.java:340)
	at javax.swing.plaf.synth.SynthGraphicsUtils.paintText(SynthGraphicsUtils.java:319)
	at javax.swing.plaf.synth.SynthGraphicsUtils.paintText(SynthGraphicsUtils.java:410)
	at javax.swing.plaf.synth.SynthLabelUI.paint(SynthLabelUI.java:213)
	at javax.swing.plaf.synth.SynthLabelUI.update(SynthLabelUI.java:177)
	at javax.swing.JComponent.paintComponent(JComponent.java:769)
	at javax.swing.JComponent.paint(JComponent.java:1045)
	at javax.swing.CellRendererPane.paintComponent(CellRendererPane.java:151)
	at javax.swing.plaf.synth.SynthTableUI.paintCell(SynthTableUI.java:693)
	at javax.swing.plaf.synth.SynthTableUI.paintCells(SynthTableUI.java:581)
	at javax.swing.plaf.synth.SynthTableUI.paint(SynthTableUI.java:365)
	at javax.swing.plaf.synth.SynthTableUI.update(SynthTableUI.java:276)
	at javax.swing.JComponent.paintComponent(JComponent.java:769)
	at javax.swing.JComponent.paint(JComponent.java:1045)
	at javax.swing.JComponent.paintChildren(JComponent.java:878)
	at javax.swing.JComponent.paint(JComponent.java:1054)
	at javax.swing.JComponent.paintToOffscreen(JComponent.java:5210)
	at javax.swing.BufferStrategyPaintManager.paint(BufferStrategyPaintManager.java:295)
	at javax.swing.RepaintManager.paint(RepaintManager.java:1249)
	at javax.swing.JComponent._paintImmediately(JComponent.java:5158)
	at javax.swing.JComponent.paintImmediately(JComponent.java:4969)
	at javax.swing.RepaintManager$3.run(RepaintManager.java:808)
	at javax.swing.RepaintManager$3.run(RepaintManager.java:796)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
	at javax.swing.RepaintManager.paintDirtyRegions(RepaintManager.java:796)
	at javax.swing.RepaintManager.paintDirtyRegions(RepaintManager.java:769)
	at javax.swing.RepaintManager.prePaintDirtyRegions(RepaintManager.java:718)
	at javax.swing.RepaintManager.access$1100(RepaintManager.java:62)
	at javax.swing.RepaintManager$ProcessingRunnable.run(RepaintManager.java:1677)
	at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:251)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:733)
	at java.awt.EventQueue.access$200(EventQueue.java:103)
	at java.awt.EventQueue$3.run(EventQueue.java:694)
	at java.awt.EventQueue$3.run(EventQueue.java:692)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:703)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:242)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:161)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:150)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:146)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:138)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:91)

This is OpenJDK 1.7. I think this may have something to do with rendering Korean text strings, as JOSM seems to work most of the time with Chinese and Japanese (such as with the nearby "place" node.)

Locale, maybe it's relevant

$ locale
LANG=en_US.utf8
LC_CTYPE=en_US.utf8
LC_NUMERIC=POSIX
LC_TIME=POSIX
LC_COLLATE=POSIX
LC_MONETARY=POSIX
LC_MESSAGES=en_US.UTF-8
LC_PAPER="en_US.utf8"
LC_NAME="en_US.utf8"
LC_ADDRESS="en_US.utf8"
LC_TELEPHONE="en_US.utf8"
LC_MEASUREMENT="en_US.utf8"
LC_IDENTIFICATION="en_US.utf8"
LC_ALL=

[jengelh@…]

screenshot

[bastiK]

Same Java version, but no exception here:

Build-Date: 2014-02-16 12:32:50
Revision: 6818
Is-Local-Build: true

Identification: JOSM/1.5 (6818 SVN en) Linux Ubuntu 12.10
Memory Usage: 543 MB / 1763 MB (103 MB allocated, but free)
Java version: 1.7.0_51, Oracle Corporation, OpenJDK 64-Bit Server VM
Java package: openjdk-7-jre:amd64-7u51-2.4.4-0ubuntu0.12.10.2
VM arguments: [-Djosm.home=pref2]
Dataset consistency test: No problems found

Plugin: mirrored_download (30197)

You have key name:ja in the screenshot, but there is no ​version of the way with this tag, why?

[jengelh@…]

I had first selected the Panmunjom place object (the house icon), which works. I then selected the building object, which led to the exception being raised and thus the screen update not being drawn correctly.

[Don-vip]

Looks like specific to OpenSUSE:

​http://forums.opensuse.org/showthread.php/493317-Java-font-issue

Can you try the solution from this thread?

try installing the mstt fonts (install the "fetchmsttfonts" package)

[jengelh@…]

To the best of my knowledge, I already have the MS fonts. Not through fetchmstt, but through properly managed RPM packages I keep for myself.

I know that Arial 3.00 (and also 6.85 from Win8.1) does not contain Korean glyphs; you need to switch to e.g. Arial Unicode MS or Batang. Such theory can be tested for arbitrary fonts using

xterm -fa "Arial" -fd "Arial"

and then attempting to type something, either showing the desired glyph (when using Arial Unicode) or a substitution box (using Arial).

The following recipe thus cures the JOSM/Swing exception, by forcing ArialUni onto every possible program. Doesn't look as good, but I'll figure that one out another day.

mkdir -p ~/.config/fontconfig/fonts.conf
cat >~/.config/fontconfig/fonts.conf <<EOF
<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<fontconfig>
        <alias binding="same">
                <family>sans-serif</family>
                <prefer>
                        <family>Arial Unicode MS</family>
                </prefer>
        </alias>
</fontconfig>
EOF

Swing could have solved this much nicer :/
I mean, GTK/Qt programs for example can correctly use freetype such that it automatically switches to another font until the glyph is found.

[Don-vip]

You know much more on this subject than me, could you please create a new bug on ​http://bugreport.java.com/bugreport/ and post here the review ID you will then get by mail? WIth a good description there are good chances to see a better fix in a future Java update.

Thanks.

[Don-vip]

Ticket #9894 has been marked as a duplicate of this ticket.

[Don-vip]

Ticket #10476 has been marked as a duplicate of this ticket.

[bastiK]

Ticket #10785 has been marked as a duplicate of this ticket.

[bastiK]

This should be reported, is it a bug in OpenSUSE or in Java?

[jengelh@…]

Some toolkit combinations, like GTK/QT with freetype automatically switch to different fonts with CJK glyphs as needed. xterm on the other hand does not (which is why it has two options, -fa and -fd, for example).

This definitely is a Java issue. The runtime ought to handle fonts with absent glyph positions - possibly by (a) throwing a NoGlyphHere, (b) ignoring such glyphs and not rendering it, (c) properly switching fonts. Throwing an "outofbounds" seems totally wrong.

[bastiK]

There is a file /etc/java-7-openjdk/fontconfig.properties (maybe at different location) where the Java fonts are set.

For me it contains e.g.

dialog.plain.latin-1=DejaVu Sans
#dialog.plain.latin-1.motif=LuxiSans-Regular
dialog.plain.japanese-sazanami=Sazanami Gothic
dialog.plain.japanese-vlgothic=VL PGothic
dialog.plain.japanese-ipafont=IPAPGothic
dialog.plain.korean-nanum=NanumGothic

If the fonts on the right hand side were not installed as a dependency of the openjdk package, then this would be issue with the package configuration and not with java. Could you check?

[jengelh@…]

I recently switched to openSUSE 13.2 (with openjdk 1.7.0.71), where the outOfBounds exception is no longer raised. It would seem that this issue has been fixed in Java.

For interest, the fontconfig.properties contains

dialog.0=-b&h-Lucida Sans-medium-r-normal--*-%d-*-*-p-*-iso10646-1
dialog.1=-Bitstream-Bitstream Vera Sans-medium-r-normal--*-%d-*-*-p-*-iso10646-1
dialog.2=-microsoft-Verdana-medium-r-normal--*-%d-*-*-p-*-iso10646-1
dialog.3=-monotype-Arial-medium-r-normal--*-%d-*-*-p-*-iso10646-1
dialog.4=-b&h-Luxi Sans-medium-r-normal--*-%d-*-*-p-*-iso10646-1
dialog.5=-URW-Nimbus Sans L-medium-r-normal--*-%d-*-*-p-*-iso8859-1
dialog.6=-URW-Helvetica-medium-r-normal--*-%d-*-*-p-*-iso8859-1
dialog.7=-URW-Symbol-medium-r-normal--*-%d-*-*-p-*-adobe-fontspecific

dialog.italic.0=-b&h-Lucida Sans-medium-i-normal--*-%d-*-*-p-*-iso10646-1
dialog.italic.1=-Bitstream-Bitstream Vera Sans-medium-i-normal--*-%d-*-*-p-*-iso
[...]

which fits the description at ​https://docs.oracle.com/javase/8/docs/technotes/guides/intl/fontconfig.html . With XLFD though, I can imagine that no fonts other than the listed ones ever get used.
Now I feel intrigued to reinstall the earlier JDK in a VM for the heck of it.

[bastiK]

#10785 reports this with openSUSE 13.2 (and Japanese text), so it doesn't seem to be fixed entirely. Maybe the problem only occurs for minimal installations and you already have the required fonts installed as dependency of another package.

[jengelh@…]

I was able to re-reproduce it all.

It turns out that Java creates a ~/.java/fonts/1.7.0_71/fcinfo-1-localhorse.lh-SuSE-13.2-.properties file where fontnames get cached, in the order of Fontconfig's preference.

This fcinfo….properties file contains seemingly every font there is on the system. Excerpt:

#JDK Font Configuration Generated File: *Do Not Edit*
#Wed Nov 26 16:28:35 CET 2014
cachedir.0=/var/cache/fontconfig
cachedir.1=/home/jengelh/.cache/fontconfig
cachedir.2=/home/jengelh/.fontconfig
version=1
fcversion=21101
sansserif.0.0.family=Arial Unicode MS
sansserif.0.0.file=/usr/share/fonts/truetype/arialuni.ttf
sansserif.0.1.family=Arial
sansserif.0.1.file=/usr/share/fonts/truetype/arial.ttf
(…)
sansserif.0.35.family=DejaVu Serif
sansserif.0.35.file=/usr/share/fonts/truetype/DejaVuSerif-Bold.ttf
sansserif.1.0.family=Arial Unicode MS
sansserif.1.0.file=/usr/share/fonts/truetype/arialuni.ttf
sansserif.1.1.family=Arial
sansserif.1.1.file=/usr/share/fonts/truetype/arialbd.ttf
(…repeat for bold…)
(…repeat for monospace, serif)

So even if I switch back the system to Arial (standard), Java would continue to use Arial (Unicode MS), causing the surprise in comment #15. Conversely, it would also be responsible for continuing outOfBounds exceptions even if additional fonts are installed later on, as this Java cache file apparently won't regenerate once it exists.

[stoecker]

Ticket #11068 has been marked as a duplicate of this ticket.

[stoecker]

References:

• ​https://forums.opensuse.org/showthread.php/493317-Java-font-issue
• ​https://bugs.openjdk.java.net/browse/JDK-7180898 (references ​https://bugs.openjdk.java.net/browse/JDK-7183251 which should be fixed?)

zypper in fetchmsttfonts did not solve this issue on my system. Also removing the cache file.

Did not test 13.2 Java 8 yet, but that has a the major issue of JPEG display, which is much worse. Reopening the bug until we either have a solution or openSUSE is really fixed.

[jengelh@…]

Installing fetchmsttfonts is not meant to fix this. It is a bug in the Java runtime that can only be worked around by changing fontconfig such that a font with all glyphs that you will eventually need has precedence.

[stoecker]

Hmm, I got rid of the error with following font installation. Still some glyphs undisplayed, but no more exceptions.

It seems the minimum to be fulfilled is to fix the errors of "fonts-config" tool. In /usr/sbin/fonts-config there is a function "generate_java_font_setup" which contains a list of fonts needed to be installed for Java.

adobe-cid-keyed-munhwa-fonts-20021114-345.1.6.noarch
arphic-bsmi00lp-fonts-20001125-774.1.2.noarch
arphic-fonts-20001125-774.1.2.noarch
arphic-gbsn00lp-fonts-20001125-774.1.2.noarch
baekmuk-ttf-fonts-2.1-8.1.2.noarch
cantarell-fonts-0.0.15-1.1.noarch
dejavu-fonts-2.34-2.1.2.noarch
efont-unicode-bitmap-fonts-0.4.2-223.1.2.noarch
fetchmsttfonts-11.4-15.1.4.noarch
fslsfonts-1.0.4-6.1.2.x86_64
ghostscript-fonts-other-9.06-4.1.2.noarch
ghostscript-fonts-std-9.06-4.1.2.noarch
gnu-unifont-bitmap-fonts-20080123-82.1.2.noarch
google-droid-fonts-20121204-2.1.2.noarch
intlfonts-1.2.1-10.1.2.noarch
intlfonts-euro-bitmap-fonts-1.2.1-10.1.2.noarch
intlfonts-ttf-fonts-1.2.1-10.1.2.noarch
ipa-ex-gothic-fonts-002.01-2.1.2.noarch
ipa-ex-mincho-fonts-002.01-2.1.2.noarch
ipa-gothic-fonts-003.03-4.1.2.noarch
ipa-mincho-fonts-003.03-4.1.2.noarch
ipa-pgothic-fonts-003.03-4.1.2.noarch
ipa-pmincho-fonts-003.03-4.1.2.noarch
ipa-uigothic-fonts-002.003-64.1.2.noarch
liberation-fonts-1.07.2-4.1.2.noarch
nanum-fonts-20110907-20.1.2.noarch
nanum-gothic-coding-fonts-2.0-20.1.2.noarch
sazanami-fonts-20040629-209.1.3.noarch
thai-fonts-0.4.17-9.1.3.noarch
un-fonts-1.0.20080608-8.1.2.noarch
wang-fonts-0.20040323-8.1.2.noarch
xano-mincho-fonts-20040509-8.1.2.noarch
xlsfonts-1.0.4-6.1.2.x86_64
xorg-x11-fonts-7.6-27.1.2.noarch
xorg-x11-fonts-core-7.6-27.1.2.noarch

[Don-vip]

Replying to stoecker:

Reopening the bug until we either have a solution or openSUSE is really fixed.

Do you think we can do something in JOSM?

[jengelh@…]

With the Advanced Properties dialog in JOSM, one can set a personal preference for a font. Unfortunately, this does not seem to change the fonts everywhere in JOSM. Most of the UI - title bar, dialogs, etc. - continue to use the system fonts instead. If that problem can be fixed (perhaps all it takes is calling some SetFont on the mainwindow), we would be a step closer to a feasible workaround.

[Don-vip]

I wrote something like that in the past (see alpha patch in #8334) but encountered performance issues and didn't look at it since.

[jengelh@…]

Backreference to ​https://bugzilla.suse.com/show_bug.cgi?id=916052

[stoecker]

According to that ticket it seems thus installation of these two fixes the bug.

arphic-bsmi00lp-fonts-20001125-774.1.2.noarch
arphic-gbsn00lp-fonts-20001125-774.1.2.noarch

[stoecker]

Replying to Don-vip:

Do you think we can do something in JOSM?

No. That's a java bug and must be fixed there. I doubt we can can fix it in JOSM, except we find a place, where we can catch any such exception and replace the text display with something else.

Nevertheless I'd let this bug stay open for reference some time.

[stoecker]

I'll setup a minimal openSUSE system next days and document the font usage for reference.

[anonymous]

See ​https://bugzilla.opensuse.org/show_bug.cgi?id=916052
Uninstalling texlive-arphic-fonts fixed a similar problem for me.

[stoecker]

Tested a bit:
The crashs happen when "texlive-arphic-fonts" is installed, but not "arphic-gbsn00lp-fonts" or "arphic-bsmi00lp-fonts". Either one of the normal arphic fonts must be installed or the texlive variant must not be installed.

[jengelh@…]

openSUSE makes TeX fonts (.pfa/.pfb) selectible through fontconfig/freetype since some releases, which is also something relatively new since some releases, and other distributions may still be not searching through pfb files (but just the regular pcf,ttfs).

[stoecker]

JOSM for openSUSE now has a recommended fonts package, which fixes this issue.

[stoecker]

Ticket #11296 has been marked as a duplicate of this ticket.

[Don-vip]

Ticket #12010 has been marked as a duplicate of this ticket.