"APP1 Segment is too long" when using "write coordinates to image headers"

[mmyfl]

After arranging images on a GPS trace, I want to write the result coordinates to image headers, but all I get is such error (and a broken jpeg file).

Please provide any additional information below. Attach a screenshot if possible.

Standard error:

ERROR: Error reading EXIF from file: com.drew.metadata.MetadataException: Tag 'GPS Latitude' has not been set -- check using containsTag() first
java.io.IOException: Write error: org.apache.commons.imaging.formats.jpeg.exif.ExifRewriter$ExifOverflowException: APP1 Segment is too long: 65751
        at org.openstreetmap.josm.plugins.photo_geotagging.ExifGPSTagger.setExifGPSTag(ExifGPSTagger.java:48)
        at org.openstreetmap.josm.plugins.photo_geotagging.GeotaggingAction$GeoTaggingRunnable.realRun(GeotaggingAction.java:231)
        at org.openstreetmap.josm.gui.PleaseWaitRunnable.doRealRun(PleaseWaitRunnable.java:93)
        at org.openstreetmap.josm.gui.PleaseWaitRunnable.run(PleaseWaitRunnable.java:161)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)
Caused by: org.apache.commons.imaging.formats.jpeg.exif.ExifRewriter$ExifOverflowException: APP1 Segment is too long: 65751
        at org.apache.commons.imaging.formats.jpeg.exif.ExifRewriter.writeSegmentsReplacingExif(ExifRewriter.java:548)
        at org.apache.commons.imaging.formats.jpeg.exif.ExifRewriter.updateExifMetadataLossless(ExifRewriter.java:394)
        at org.apache.commons.imaging.formats.jpeg.exif.ExifRewriter.updateExifMetadataLossless(ExifRewriter.java:294)
        at org.openstreetmap.josm.plugins.photo_geotagging.ExifGPSTagger.setExifGPSTagWorker(ExifGPSTagger.java:146)
        at org.openstreetmap.josm.plugins.photo_geotagging.ExifGPSTagger.setExifGPSTag(ExifGPSTagger.java:44)
        ... 6 more

Repository Root: http://josm.openstreetmap.de/svn
Build-Date: 2015-08-06 01:32:57
Last Changed Author: wiktorn
Revision: 8643
Repository UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Relative URL: ^/trunk
URL: http://josm.openstreetmap.de/svn/trunk
Last Changed Date: 2015-08-05 23:51:30 +0200 (Wed, 05 Aug 2015)
Last Changed Rev: 8643

Identification: JOSM/1.5 (8643 en) Windows XP 32-Bit
Memory Usage: 122 MB / 247 MB (12 MB allocated, but free)
Java version: 1.7.0_80, Oracle Corporation, Java HotSpot(TM) Client VM
Program arguments: [--debug]

Plugins:
- InfoMode (31241)
- commons-imaging (31241)
- photo_geotagging (31241)
- reverter (31241)
- undelete (31241)
- utilsplugin2 (31241)

Last errors/warnings:
- E: Error reading EXIF from file: com.drew.metadata.MetadataException: Tag 'GPS Latitude' has not been set -- check using containsTag() first

[bastiK]

Please attach a sample image (before write attempt by JOSM plugin).

[mmyfl]

Replying to bastiK:

Please attach a sample image (before write attempt by JOSM plugin).

Wow, I found the reason: The file is just too big (2.3 MB). Resizing seems to solve this issue.
Here is a sample as you requested: ​http://pan.baidu.com/s/1ntCaeAt

[bastiK]

Replying to mmyfl:

Replying to bastiK:

Please attach a sample image (before write attempt by JOSM plugin).

Wow, I found the reason: The file is just too big (2.3 MB).

2.3 MB is a normal size for an image file, this cannot be the reason.

Resizing seems to solve this issue.

My guess is that resizing changed the EXIF-header, so the plugin had no problem with it afterwards.

Here is a sample as you requested: ​http://pan.baidu.com/s/1ntCaeAt

Thanks, I'll have a look at it.

[simon04]

I added a unit test for one of the images in [o31994] and created a ticket in the Commons Imaging project: ​https://issues.apache.org/jira/browse/IMAGING-179

[simon04]

​https://issues.apache.org/jira/browse/IMAGING-179?focusedCommentId=15469184&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15469184

The way that lossless rewriting currently works is we blacklist every EXIF byte, and then traverse the directory structure, whitelisting bytes occupied by known tags. Every whitelisted byte can then be overwritten, since its contents are already known and can be rewritten.
Since images can have tags with unknown contents (such as this particular image's enormous "Maker Note" tag 927c using up 01008-35518) we can't whitelist any bytes that appear unused, as those tags can contain anything, including arbitrary numbers of recursive references to other directories and their tags, using up space elsewhere in the EXIF data, and can even be encrypted.
That only leaves growing the EXIF field, but this one is already using 65526 bytes, leaving only another 9 bytes to use, which is not enough.
As a workaround you could try save space by deleting the thumbnail, or, if you don't care about unknown tags, use lossy rewriting instead (which can corrupt them).
We would have to parse the "Maker Note" to extract more space. This is manufacturer-specific :-/.

[Bjoeni]

I'm currently writing a patch for this allowing the user to try the lossy approach on images where the lossless approach failed, but it still depends on #20341 (obviously not directly depending but conflicting, so I'm gonna wait till that's committed)

[Bjoeni]

In 35738/osm:

fix #11757

• Allow user to try tagging images using the lossy approach in case of ExifOverflowException
• Automatically restore backup file in case something goes wrong

[Bjoeni]

In 35777/osm:

see #11757, #21019 - add GeotaggingActionTest