EXIF question asked with all images one by one

[kempelen]

Hi,
The attached window does not allow "Yes to all", so it repeats the question on all files with my camera.

off: can I somehow change my image to avoid this?

"The GPS tag could not be added to the file XX.JPG because there is not enough free space in the EXIF section.
This can likely be fixed by rewriting the entire EXIF section, however some metadata may get lost in the process.
Would you like to try again using the lossy approach?"

[Bjoeni]

(introduced with #11757)

[Bjoeni]

In 35776/osm:

fix #21019 - ask only once for photos that failed because of an ExifOverflowException

[Bjoeni]

In 35777/osm:

see #11757, #21019 - add GeotaggingActionTest

[Bjoeni]

In 35778/osm:

see #21019 - dist

[Bjoeni]

(I'm aware that according to Jenkins that the dist is broken, I'll fix it)

[Bjoeni]

Actually, no it's probably not. I think there's something wrong with the seachart plugin that was updated in r35773/osm

Could someone who knows more about that confirm if that's actually the cause?

/tmp/jenkins7694176884346300550.sh: line 20: cd: /var/lib/jenkins/jobs/JOSM-Plugins/workspace/jdk/JDK8/SeaChart: No such file or directory
Build step 'Execute shell' marked build as failure
New run name is '35778'
Started calculate disk usage of build
Finished Calculation of disk usage of build in 0 seconds
Started calculate disk usage of workspace
Finished Calculation of disk usage of workspace in 0 seconds
[WS-CLEANUP] Deleting project workspace...
[WS-CLEANUP] Deferred wipeout is used...
[WS-CLEANUP] done
Finished: FAILURE

[kempelen]

Thank you for the wonderfully fast fix!

Can somebody tell me if this original error can be avoided by making something with the EXIF of the images in advance, e.g. doing something with the exiv2 tool?

That would avoid "lossy approach" - so I'd prefer that.

[Bjoeni]

Probably not really as the Apache Commons Imaging library that is used can already understand many tags, the problem are mostly manufacturer-specific tags that are sometimes not even properly documented. And I don't think I ever lost a tag that was useful to me.

See this ​explanation how rewriting is done:

The way that lossless rewriting currently works is we blacklist every EXIF byte, and then traverse the directory structure, whitelisting bytes occupied by known tags. Every whitelisted byte can then be overwritten, since its contents are already known and can be rewritten.

Since images can have tags with unknown contents (such as this particular image's enormous "Maker Note" tag 927c using up 01008-35518) we can't whitelist any bytes that appear unused, as those tags can contain anything, including arbitrary numbers of recursive references to other directories and their tags, using up space elsewhere in the EXIF data, and can even be encrypted.

That only leaves growing the EXIF field, but this one is already using 65526 bytes, leaving only another 9 bytes to use, which is not enough.

Here you can see that (at least in my case) not a single tag was lost that could be read by ExifTool. And as you can see based on the file size, despite that it actually got smaller. So all that was lost were some Samsung-specific tags that could e.g. contain information about how a panorama was taken. But unless you know exactly how they are written you can't save them.

before

after

ExifTool Version Number : 12.14 File Name : test (1).jpg Directory : . File Size : 4.2 MiB File Modification Date/Time : 2019:05:19 22:42:40+02:00 File Access Date/Time : 2021:07:14 17:16:00+02:00 File Creation Date/Time : 2021:07:14 11:55:42+02:00 File Permissions : rw-rw-rw- File Type : JPEG File Type Extension : jpg MIME Type : image/jpeg Exif Byte Order : Little-endian (Intel, II) Orientation : Horizontal (normal) Y Cb Cr Positioning : Centered X Resolution : 72 Y Resolution : 72 Resolution Unit : inches Make : samsung Camera Model Name : SM-G965F Software : G965FXXS3CSD1 Modify Date : 2019:05:19 17:42:39 Exposure Time : 1/855 F Number : 2.4 Exposure Program : Program AE ISO : 50 Exif Version : 0220 Date/Time Original : 2019:05:19 17:42:39 Create Date : 2019:05:19 17:42:39 Shutter Speed Value : 1/855 Aperture Value : 2.4 Brightness Value : 8.13 Exposure Compensation : 0 Max Aperture Value : 1.5 Metering Mode : Spot Flash : No Flash Flashpix Version : 0100 Components Configuration : Y, Cb, Cr, - Focal Length : 4.3 mm Sub Sec Time : 0943 Sub Sec Time Original : 0943 Sub Sec Time Digitized : 0943 User Comment : Color Space : sRGB Exif Image Width : 4032 Exif Image Height : 3024 Scene Type : Unknown (.) Custom Rendered : Normal Exposure Mode : Auto White Balance : Auto Digital Zoom Ratio : undef Focal Length In 35mm Format : 26 mm Scene Capture Type : Standard Contrast : Normal Saturation : Normal Sharpness : Normal Image Unique ID : I12LLKF00SM Interoperability Index : R98 - DCF basic file (sRGB) Interoperability Version : 0100 Compression : JPEG (old-style) Thumbnail Offset : 911 Thumbnail Length : 21562 Image Width : 4032 Image Height : 3024 Encoding Process : Baseline DCT, Huffman coding Bits Per Sample : 8 Color Components : 3 Y Cb Cr Sub Sampling : YCbCr4:2:0 (2 2) Time Stamp : 2019:05:19 23:42:40+02:00 Aperture : 2.4 Image Size : 4032x3024 Megapixels : 12.2 Scale Factor To 35 mm Equivalent: 6.0 Shutter Speed : 1/855 Create Date : 2019:05:19 17:42:39.0943 Date/Time Original : 2019:05:19 17:42:39.0943 Modify Date : 2019:05:19 17:42:39.0943 Thumbnail Image : (Binary data 21562 bytes, use -b option to extract) Circle Of Confusion : 0.005 mm Field Of View : 69.4 deg Focal Length : 4.3 mm (35 mm equivalent: 26.0 mm) Hyperfocal Distance : 1.55 m Light Value : 13.3

ExifTool Version Number : 12.14 File Name : test (1).jpg Directory : . File Size : 4.1 MiB File Modification Date/Time : 2019:05:19 18:42:39+02:00 File Access Date/Time : 2021:07:14 17:25:53+02:00 File Creation Date/Time : 2021:07:14 17:25:38+02:00 File Permissions : rw-rw-rw- File Type : JPEG File Type Extension : jpg MIME Type : image/jpeg Exif Byte Order : Little-endian (Intel, II) Make : samsung Camera Model Name : SM-G965F Orientation : Horizontal (normal) X Resolution : 72 Y Resolution : 72 Resolution Unit : inches Software : G965FXXS3CSD1 Modify Date : 2019:05:19 17:42:39 Y Cb Cr Positioning : Centered Exposure Time : 1/855 F Number : 2.4 Exposure Program : Program AE ISO : 50 Exif Version : 0220 Date/Time Original : 2019:05:19 17:42:39 Create Date : 2019:05:19 17:42:39 Components Configuration : Y, Cb, Cr, - Shutter Speed Value : 1/855 Aperture Value : 2.4 Brightness Value : 8.13 Exposure Compensation : 0 Max Aperture Value : 1.5 Metering Mode : Spot Flash : No Flash Focal Length : 4.3 mm User Comment : Sub Sec Time : 0943 Sub Sec Time Original : 0943 Sub Sec Time Digitized : 0943 Flashpix Version : 0100 Color Space : sRGB Exif Image Width : 4032 Exif Image Height : 3024 Interoperability Index : R98 - DCF basic file (sRGB) Interoperability Version : 0100 Scene Type : Unknown (.) Custom Rendered : Normal Exposure Mode : Auto White Balance : Auto Digital Zoom Ratio : undef Focal Length In 35mm Format : 26 mm Scene Capture Type : Standard Contrast : Normal Saturation : Normal Sharpness : Normal Image Unique ID : I12LLKF00SM GPS Version ID : 2.3.0.0 GPS Latitude Ref : North GPS Longitude Ref : East GPS Time Stamp : 16:42:39 GPS Date Stamp : 2019:05:19 Compression : JPEG (old-style) Thumbnail Offset : 948 Thumbnail Length : 21562 Image Width : 4032 Image Height : 3024 Encoding Process : Baseline DCT, Huffman coding Bits Per Sample : 8 Color Components : 3 Y Cb Cr Sub Sampling : YCbCr4:2:0 (2 2) Time Stamp : 2019:05:19 23:42:40+02:00 Aperture : 2.4 Image Size : 4032x3024 Megapixels : 12.2 Scale Factor To 35 mm Equivalent: 6.0 Shutter Speed : 1/855 Create Date : 2019:05:19 17:42:39.0943 Date/Time Original : 2019:05:19 17:42:39.0943 Modify Date : 2019:05:19 17:42:39.0943 Thumbnail Image : (Binary data 21562 bytes, use -b option to extract) GPS Date/Time : 2019:05:19 16:42:39Z GPS Latitude : (censored) N GPS Longitude : (censored) E Circle Of Confusion : 0.005 mm Field Of View : 69.4 deg Focal Length : 4.3 mm (35 mm equivalent: 26.0 mm) GPS Position : (censored) N, (censored) E Hyperfocal Distance : 1.55 m Light Value : 13.3

Maybe we could make it more clear that almost all tags will be kept (just restructured). But we can't be sure about all tags, that's why I decided to show a warning.

[anonymous]

Thank you for the detailed explanation Bjoeni. Then I think it's all OK as it is now.

Maybe I was more worried than necessary because 5+ years ago this plugin somtimes very badly destroyed EXIF.

[Bjoeni]

In 35783/osm:

see #21019 - change warning text for ExifOverflowException

[Bjoeni]

In 35784/osm:

see #21019 - dist

[Bjoeni]

I changed the dialog text to make it less "frightening":

It still clearly warns about the fact that tags could get lost but hopefully it's a bit more clear that usually everything should be fine.

[taylor.smock]

@Bjoeni: I'm currently working on converting the JOSM plugin repo to git (see #23286). Do you have a preferred name/email for that?

[taylor.smock]

@Bjoeni: Poke: If you need/want the patches you've contributed to JOSM plugins (via svn) to be trackable on git forges like GitHub or GitLab, I do need to know what email to use for that.

As a specific example, the git conversion has r35783/osm as

Author: Bjoeni <Bjoeni@svn.openstreetmap.org>
Commit: Bjoeni <Bjoeni@svn.openstreetmap.org>

    see #21019 - change warning text for ExifOverflowException

I do have a deadline of early January to get the proper attribution information for svn committers and patch providers (see ​https://github.blog/2023-01-20-sunsetting-subversion-support/ ).

[Bjoeni]

@Taylor: sorry for not getting back to you earlier, I sent you an email.