Merge pull request #64 from mattpopovich/adobe-dng-converter-cli

Finalize Adobe DNG Converter post
mattpopovich · Feb 27, 2024 · b960cf9 · b960cf9
2 parents 0739ee8 + 718447a
commit b960cf9
Showing 1 changed file with 66 additions and 39 deletions.
diff --git a/_posts/2024-02-23-how-to-use-adobe-dng-converter-from-the-command-line.md b/_posts/2024-02-23-how-to-use-adobe-dng-converter-from-the-command-line.md
@@ -23,7 +23,7 @@ mermaid: false              # Diagram generation tool via ```mermaid [...]```
 <!-- TODO: Link to "my FCPX export settings blog post. Start this blog with "expanding on my previous project" -->
 
 ## Intro
-Whenever I shoot a time lapse in raw on my [GoPro (affiliate link)](https://amzn.to/3ZUuXcD), it creates one folder that is full of `*.JPG` and `*.GPR` files. I then find myself moving all the `*.JPG` files into a `JPG` folder, and all the `*.GPR` files into a `GPR` folder. I then use [Adobe Digital Negative (DNG) Converter](https://helpx.adobe.com/camera-raw/using/adobe-dng-converter.html) to convert the `.GPR` files into `.DNG` so that I can import them to [Final Cut Pro (FCP/FCPX)](https://www.apple.com/final-cut-pro/). Adobe DNG Converter is a [graphical user interface (GUI)](https://www.computerhope.com/jargon/g/gui.htm) program which means I need to open the program, select input and ouput destinations, select my seetings, all while using the mouse to make my selections. It's slow and repetive (not to mention I make the same selections every time). I just want something that I can double click that will move the pictures to their respective folders and start the `GPR` to `DNG` conversion process. Turns out, as I suspected, the Adobe DNG Converter is running a command line program underneath the hood. Here's how we can use it without the GUI to "automate" away this repetitive task:
+Whenever I shoot a time lapse in raw on my [GoPro (affiliate link)](https://amzn.to/3ZUuXcD), it creates one folder that is full of `*.JPG` and `*.GPR` files. I then find myself moving all the `*.JPG` files into a `JPG` folder, and all the `*.GPR` files into a `GPR` folder. I then use [Adobe Digital Negative (DNG) Converter](https://helpx.adobe.com/camera-raw/using/adobe-dng-converter.html) to convert the `.GPR` files into `.DNG` so that I can import them to [Final Cut Pro (FCP/FCPX)](https://www.apple.com/final-cut-pro/). Adobe DNG Converter is a [graphical user interface (GUI)](https://www.computerhope.com/jargon/g/gui.htm) program which means I need to open the program, select input and output destinations, select my settings, all while using the mouse to make my selections. It's slow and repetitive (not to mention I make the same selections every time). I just want something that I can double click that will move the pictures to their respective folders and start the `GPR` to `DNG` conversion process. Turns out, as I suspected, the Adobe DNG Converter is running a command line program underneath the hood. Here's how we can use it without the GUI to "automate" away this repetitive task:
 
 > Note that this article assumes you have basic familiarity with the terminal and shell commands
 {: .prompt-warning }
@@ -38,7 +38,9 @@ Where
   * `default is sequential (one image at a time)`
 * `-d = output directory`
 
-Additional arguments can be found in [this .pdf](https://helpx.adobe.com/content/dam/help/en/camera-raw/digital-negative/jcr_content/root/content/flex/items/position/position-par/download_section/download-1/dng_converter_commandline.pdf)
+Additional arguments can be found in [this .pdf](https://helpx.adobe.com/content/dam/help/en/camera-raw/digital-negative/jcr_content/root/content/flex/items/position/position-par/download_section/download-1/dng_converter_commandline.pdf).
+
+View the scripts I made for this post at my GitHub repo [*AdobeDNGConverterScripts*](https://github.com/mattpopovich/AdobeDNGConverterScripts).
 
 ## Installation
 Go [here](https://helpx.adobe.com/camera-raw/using/adobe-dng-converter.html), download and install the application.
@@ -75,7 +77,7 @@ total 551496
 -rwxr-xr-x  1 root  wheel  282361984 Jun  1  2023 Adobe DNG Converter
 ```
 
-I'm not sure how to bring up a help menu for it... The CLI for Adobe DNG Converter has limited documentation available. [This .pdf](https://helpx.adobe.com/content/dam/help/en/camera-raw/digital-negative/jcr_content/root/content/flex/items/position/position-par/download_section/download-1/dng_converter_commandline.pdf) is about all I can find. Thankfully it explains some of the acceptable arguments that the executable understands.
+I'm not sure how to bring up a help menu for it... The CLI for Adobe DNG Converter has limited documentation available. [This .pdf](https://helpx.adobe.com/content/dam/help/en/camera-raw/digital-negative/jcr_content/root/content/flex/items/position/position-par/download_section/download-1/dng_converter_commandline.pdf) is about all I can find. Thankfully, that `.pdf` explains some of the acceptable arguments that the executable understands.
 
 If you want the same default settings that the GUI provides, you can use the following command:
 ```console
@@ -95,6 +97,31 @@ matt@mac $ /Applications/Adobe\ DNG\ Converter.app/Contents/MacOS/Adobe\ DNG\ Co
 * `-mp = process multiple files in parallel`
   * `default is sequential (one image at a time)`
 
+## My GoPro Organization Script
+[I wrote a script](https://github.com/mattpopovich/AdobeDNGConverterScripts/blob/main/organizeGoProDNG.sh) that will convert take my GoPro's standard RAW timelapse output:
+* `GOPR0000.GPR`
+* `GOPR0000.JPG`
+* `GOPR0001.GPR`
+* `GOPR0001.JPG`
+
+and convert it to:
+* `GPR/`
+  * `GOPR0000.GPR`
+  * `GOPR0001.GPR`
+* `JPG/`
+  * `GOPR0000.JPG`
+  * `GOPR0001.JPG`
+* `DNG/`
+  * `GOPR0000.dng`
+  * `GOPR0001.dng`
+
+You can place [organizeGoProDNG.sh](https://github.com/mattpopovich/AdobeDNGConverterScripts/blob/main/organizeGoProDNG.sh) into the folder with your GoPro's images and run it to begin the organization and conversion.
+
+**Pro tip**: If you change the extension of the script from `.sh` to `.command`, you can "double click" it in Finder to run it!
+
+> **DO NOT** run random scripts from the internet, especially from untrusted parties. Please review scripts before running them. Even if you don't understand the script, it's worth pasting it into a large language model (LLM) such as [ChatGPT](https://chat.openai.com/) and asking it to explain the script for you, ask if there is anything dangerous or malicious in it, etc.
+{: .prompt-danger }
+
 ## Advanced information
 
 ### Reproducibility
@@ -103,63 +130,63 @@ Note that these files are not bitwise reproducible (but they are exactly the sam
 matt@mac $ /Applications/Adobe\ DNG\ Converter.app/Contents/MacOS/Adobe\ DNG\ Converter -fl *.GPR
 matt@mac $ /Applications/Adobe\ DNG\ Converter.app/Contents/MacOS/Adobe\ DNG\ Converter -fl *.GPR
 matt@mac $ shasum -a 256 *.dng
-486395fba712ddaf3265d503f53a21a0977401e95e935857863239017692d49f  GOPR0924.dng
-66a4c517d62a396977cf101e49026e0dc2d4f9ab0775a6dd96b81e5dcd41daaa  GOPR0924_1.dng
-5c647d26c98bbcb34f66c7c8530cc006fab6a716d13f42dc98fa63ddb1b0b3e7  GOPR0925.dng
-2864621bea9c221750bcea31b52009a063a36ebfd025bc892267e20a22c3986f  GOPR0925_1.dng
+486395fba712ddaf3265d503f53a21a0977401e95e935857863239017692d49f  GOPR0000.dng
+66a4c517d62a396977cf101e49026e0dc2d4f9ab0775a6dd96b81e5dcd41daaa  GOPR0000_1.dng
+5c647d26c98bbcb34f66c7c8530cc006fab6a716d13f42dc98fa63ddb1b0b3e7  GOPR0001.dng
+2864621bea9c221750bcea31b52009a063a36ebfd025bc892267e20a22c3986f  GOPR0001_1.dng
 matt@mac $ ls -l *.dng
--rw-r--r--  1 matt  staff  15166544 Feb 30 03:17 GOPR0924.dng
--rw-r--r--  1 matt  staff  15166544 Feb 30 03:17 GOPR0924_1.dng
--rw-r--r--  1 matt  staff  14433870 Feb 30 03:17 GOPR0925.dng
--rw-r--r--  1 matt  staff  14433870 Feb 30 03:17 GOPR0925_1.dng
+-rw-r--r--  1 matt  staff  11876742 Feb 26 22:55 GOPR0000.dng
+-rw-r--r--  1 matt  staff  11876742 Feb 26 22:55 GOPR0000_1.dng
+-rw-r--r--  1 matt  staff  13340838 Feb 26 22:55 GOPR0001.dng
+-rw-r--r--  1 matt  staff  13340838 Feb 26 22:55 GOPR0001_1.dng
 ```
 
 ### Argument runtime and file size comparison
 
 I am using 3000x4000 `.GPR` (raw) files shot with my [GoPro](https://amzn.to/3ZUuXcD) Hero 8 Black:
 ```console
-matt@mac $ file GPR/*
-GPR/GOPR0924.GPR: TIFF image data, little-endian, direntries=57, height=3000, bps=16, compression=JBIG, ITU-T T.85, PhotometricIntepretation=(unknown=0xffff8023), description=C:\DCIM\100GOPRO\GOPR0924.GPR, manufacturer=GoPro, model=HERO8 Black, orientation=[*2*], width=4000
+matt@mac $ file *.GPR
+GOPR0000.GPR: TIFF image data, little-endian, direntries=58, height=3000, bps=16, compression=JBIG, ITU-T T.85, PhotometricIntepretation=(unknown=0xffff8023), description=C:\DCIM\100GOPRO\GOPR3115.GPR, manufacturer=GoPro, model=HERO8 Black, orientation=[*2*], width=4000
 
-GPR/GOPR0925.GPR: TIFF image data, little-endian, direntries=58, height=3000, bps=16, compression=JBIG, ITU-T T.85, PhotometricIntepretation=(unknown=0xffff8023), description=C:\DCIM\100GOPRO\GOPR0925.GPR, manufacturer=GoPro, model=HERO8 Black, orientation=[*2*], width=4000
+GOPR0001.GPR: TIFF image data, little-endian, direntries=58, height=3000, bps=16, compression=JBIG, ITU-T T.85, PhotometricIntepretation=(unknown=0xffff8023), description=C:\DCIM\100GOPRO\G0029214.GPR, manufacturer=GoPro, model=HERO8 Black, orientation=[*2*], width=4000
 
-matt@mac $ ls -l GPR
--rwxr-xr-x  1 mattpopovich  staff  3926850 Feb 23 18:19 GOPR0924.GPR
--rwxr-xr-x  1 mattpopovich  staff  3492368 Feb 23 18:19 GOPR0925.GPR
+matt@mac $ ls -l *.GPR
+-rwx------  1 matt  staff  6186776 Feb 26 22:54 GOPR0000.GPR
+-rwx------  1 matt  staff  3677656 Feb 26 22:54 GOPR0001.GPR
 ```
 
 <details markdown="1">
-  <summary><a href="https://github.com/mattpopovich/AdobeDNGConverterScripts/blob/main/compare_adobe_dng_converter_arguments.sh">I wrote a script</a> to compare the runtime and output file sizes of the different arguments for the executable</summary>
+  <summary><a href="https://github.com/mattpopovich/AdobeDNGConverterScripts/blob/main/compare_AdobeDNGConverter_arguments.sh">I wrote a script</a> to compare the runtime and output file sizes of the different arguments for the executable</summary>
   ```console
-  matt@mac $ ./compareDNGsettings.sh
-  Default = 28492484B, .8896226s
-  Using flags -c -p1        -cr7.1 -dng1.7.1 = -.0081965s (0%), +0B (+0%) vs default
-  Using flags -u -p1        -cr7.1 -dng1.7.1 = -.0341326s (-3%), +0B (+0%) vs default
-  Using flags -l -p1        -cr7.1 -dng1.7.1 = +.0024795s (+0%), +0B (+0%) vs default
-  Using flags -c -p1 -e     -cr7.1 -dng1.7.1 = -.0055936s (0%), +0B (+0%) vs default
-  Using flags -c -p0        -cr7.1 -dng1.7.1 = -.0499066s (-5%), -238840B (0%) vs default
-  Using flags -c -p2        -cr7.1 -dng1.7.1 = +.7349366s (+82%), +4165366B (+14%) vs default
-  Using flags -c -p1 -fl    -cr7.1 -dng1.7.1 = +.0102043s (+1%), +1379742B (+4%) vs default
-  Using flags -c -p1 -lossy -cr7.1 -dng1.7.1 = +.1604536s (+18%), -18143172B (-63%) vs default
-  Using flags -c -p1 -mp    -cr7.1 -dng1.7.1 = -.2128286s (-23%), +0B (+0%) vs default
+  matt@mac $ ./compare_AdobeDNGConverter_arguments.sh
+  Default = 25217580B, .992175s
+  Using flags -c -p1        -cr7.1 -dng1.7.1 = -.0003828s (0%), +0B (+0%) vs default
+  Using flags -u -p1        -cr7.1 -dng1.7.1 = -.0146741s (-1%), +0B (+0%) vs default
+  Using flags -l -p1        -cr7.1 -dng1.7.1 = -.0108651s (-1%), +0B (+0%) vs default
+  Using flags -c -p1 -e     -cr7.1 -dng1.7.1 = -.002032s (0%), +0B (+0%) vs default
+  Using flags -c -p0        -cr7.1 -dng1.7.1 = -.066626s (-6%), -140136B (0%) vs default
+  Using flags -c -p2        -cr7.1 -dng1.7.1 = +.7514072s (+75%), +3101062B (+12%) vs default
+  Using flags -c -p1 -fl    -cr7.1 -dng1.7.1 = +.043321s (+4%), +2295970B (+9%) vs default
+  Using flags -c -p1 -lossy -cr7.1 -dng1.7.1 = +.2135278s (+21%), -8252052B (-32%) vs default
+  Using flags -c -p1 -mp    -cr7.1 -dng1.7.1 = -.3165521s (-31%), +0B (+0%) vs default
   ```
 </details>
 
-This particular example was ran using the two `*.GPR` files I mentioned above.
+This particular example was ran using the two `*.GPR` files I mentioned above (available from [my GitHub repo](https://github.com/mattpopovich/AdobeDNGConverterScripts/tree/main/examples)).
 
 <!-- Had to add <code> &nbsp; </code> here because spaces are not kept between `` unfortunately -->
 
 | [Flags](https://helpx.adobe.com/content/dam/help/en/camera-raw/digital-negative/jcr_content/root/content/flex/items/position/position-par/download_section/download-1/dng_converter_commandline.pdf) | Speedup (s) | Speedup (%) | Size (bytes) | Size (%) |
 |-------|-------------|-------------|--------------|----------|
-| <code>-c -p1 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -cr7.1 -dng1.7.1</code> | -.0081965s | 0% | +0B | +0% |
-| <code>-u -p1 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -cr7.1 -dng1.7.1</code> | -.0341326s | -3% | +0B | +0% |
-| <code>-l -p1 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -cr7.1 -dng1.7.1</code> | +.0024795s | +0% | +0B | +0% |
-| <code>-c -p1 -e &nbsp;&nbsp;&nbsp; -cr7.1 -dng1.7.1</code> | -.0055936s | 0% | +0B | +0% |
-| <code>-c -p0 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -cr7.1 -dng1.7.1</code> | -.0499066s | -5% | -238,840B | 0% |
-| <code>-c -p2 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -cr7.1 -dng1.7.1</code> | +.7349366s | +82% | +4,165,366B | +14% |
-| <code>-c -p1 -fl &nbsp;&nbsp; -cr7.1 -dng1.7.1</code> | +.0102043s | +1% | +1,379,742B | +4% |
-| <code>-c -p1 -lossy -cr7.1 -dng1.7.1</code> | +.1604536s | +18% | -18,143,172B | -63% |
-| <code>-c -p1 -mp &nbsp;&nbsp; -cr7.1 -dng1.7.1</code> | -.2128286s | -23% | +0B | +0% |
+| <code>-c -p1 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -cr7.1 -dng1.7.1</code> | -.0003828s | 0%   | +0B         | +0% |
+| <code>-u -p1 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -cr7.1 -dng1.7.1</code> | -.0146741s | -1%  | +0B         | +0% |
+| <code>-l -p1 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -cr7.1 -dng1.7.1</code> | -.0108651s | -1%  | +0B         | +0% |
+| <code>-c -p1 -e &nbsp;&nbsp;&nbsp; -cr7.1 -dng1.7.1</code>                | -.002032s  | 0%   | +0B         | +0% |
+| <code>-c -p0 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -cr7.1 -dng1.7.1</code> | -.066626s  | -6%  | -140,136B   | 0% |
+| <code>-c -p2 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -cr7.1 -dng1.7.1</code> | +.7514072s | +75% | +3,101,062B | +14% |
+| <code>-c -p1 -fl &nbsp;&nbsp; -cr7.1 -dng1.7.1</code>                     | +.043321s  | +4%  | +2,295,970B | +4% |
+| <code>-c -p1 -lossy -cr7.1 -dng1.7.1</code>                               | +.2135278s | +21% | -8,252,052B | -63% |
+| <code>-c -p1 -mp &nbsp;&nbsp; -cr7.1 -dng1.7.1</code>                     | -.3165521s | -31% | +0B         | +0% |
 
 Where
 * Speedup = smaller (negative) is better