id summary reporter owner description type status priority milestone component version resolution keywords cc 15574 [patch] [experimental] make large jpeg loading work through the use of JNI bridge to turbojpeg system library cmuelle8 team "Image loading in JOSM is currently done with {{{ #!java Image img = Toolkit.getDefaultToolkit().createImage(file.getPath()); [..] tracker.addImage(img, 1); }}} This may cause issues when trying to load images with large dimensions, if the heap space cannot hold the buffer of a full decode of the image. JPEG loading however may be done in a way that the native decoding library decodes and scales the image on the fly (before the pixels are being sent to a BufferedImage). JDK8 and JDK9 already ''use'' jpeg or turbojpeg system libraries in the background to do the heavy lifting, but there does not seem to be a proper way in those JDKs to delegate the scaling wish down to the JNI level. The imageio framework has {{{canSetSourceRenderSize()}}} to check if a plugin can handle a custom source render size, but this is currently (JDK7, JDK8, JDK9) not implemented for the JPEG decoder, see * https://docs.oracle.com/javase/9/docs/api/javax/imageio/ImageReadParam.html#canSetSourceRenderSize-- * https://docs.oracle.com/javase/9/docs/api/index.html?javax/imageio/plugins/jpeg/JPEGImageReadParam.html ----- One solution is to employ the JNI bridge to turbojpeg system library. * https://libjpeg-turbo.org/Documentation/Documentation * https://cdn.rawgit.com/libjpeg-turbo/libjpeg-turbo/dev/java/doc/index.html Problems (caveats) with this approach are * libjpeg-turbo on the host system must have been compiled with the {{{--with-java}}} swith to {{{./configure}}} * debian/ubuntu precompiled versions do not have java enabled by default, see [https://anonscm.debian.org/cgit/collab-maint/libjpeg-turbo.git/tree/debian/rules debian/rules] * the library needs to be found and readable by the JVM to load (general JNI problem) * as the loaded image will be a scaled version, zooming in 1:1 will not be possible * this may be overcome by reloading subregions of the source image from file as needed * ~~in principle this should be possible using [https://cdn.rawgit.com/libjpeg-turbo/libjpeg-turbo/dev/java/doc/org/libjpegturbo/turbojpeg/TJDecompressor.html#decompress(int%5B%5D,%20int,%20int,%20int,%20int,%20int,%20int,%20int) this decompress variant] instead of the convenience function~~ * not true currently: this variant merely enables to write to another offset than (0,0) in the dest buffer * while partial jpeg decoding [https://github.com/libjpeg-turbo/libjpeg-turbo/blob/19b393b62422c97d97de4a5f2943d2cf4dc44120/ChangeLog.md#1490-15-beta1 was added in turbojpeg 1.5] it has [https://github.com/libjpeg-turbo/libjpeg-turbo/blob/19b393b62422c97d97de4a5f2943d2cf4dc44120/turbojpeg-jni.c#L630 not been promoted to the callable jni] functions yet * i.e. turbojpeg-jni.c lacks sth. like {{{TJDecompressor_decompress_partial}}} that employs [https://github.com/libjpeg-turbo/libjpeg-turbo/blob/5bc43c7821df982f65aa1c738f67fbf7cba8bd69/jdapistd.c#L143 jpeg_crop_scanline(..)] and [https://github.com/libjpeg-turbo/libjpeg-turbo/blob/5bc43c7821df982f65aa1c738f67fbf7cba8bd69/jdapistd.c#L361 jpeg_skip_scanlines(..)] * once this is added to the jni interface of turbojpeg it might be feasible to load the visible subregions (resulting from zoom or drag operations) dynamically from a large jpeg file as needed ----- Test data (a jpeg image with 30.000x21.952 pixels) may be found under * https://commons.wikimedia.org/wiki/File:Pieter_Bruegel_the_Elder_-_The_Tower_of_Babel_(Vienna)_-_Google_Art_Project_-_edited.jpg Trying to load this file without the proposed patch results in * in {{{java.lang.OutOfMemoryError: Java heap space}}} or * if JOSM is started with {{{-Xmx4G}}} (or a variant) to supply more heap space * in a terminated JVM and a segfaulting Xorg (libpthread related) * serious bug related to OOM conditions in openjdk (?) ----- The proposed patch makes an effort to determine the presumably available memory before loading an image. It reads width and height of an image without loading it and then checks * if the image will presumably fit into half of the avail memory to the JVM loading is defered to the default toolkit as usual * otherwise an instance of TJDecoder is used to load a scaled version (that will also not employ more than half of the available JVM heap space memory) Not taking more than half of the memory available is especially necessary for the bilinear scaling to work as it duplicates the image buffer (worst case) in best fit mode. In general, because of the deliberate checking for possible OOM conditions, JOSM will be from now on less likely to hit such exceptions if large images are loaded (this applies to all image formats, not just JPEG, but for JPEGs the code will make a second attempt in loading the image, instead of giving up). ----- Future work: Considering that bilinear scaling may be turned off in prefs, the constraint of not using more than half the available memory may be loosened. However, a dynamic check depending on the state bilin* options are in, might need more work, i.e. if that option is changed during runtime and an image is loaded already employing more than half of the available memory, it may seem strange to do an expensive reload of the whole image, just because this scaling option changed. So for now, this check is not dependent on the scaling option used for ImageDisplay. Pledge for further testing: This patch had limited testing on a single dev machine and a single OS. It has not been tested, whether libjpeg-turbo is properly loaded under Windows or MacOS, e.g. " enhancement new normal Core image mapping latest jpeg geoimage turbojpeg large images libjpeg-turbo jni