Damage
is a highly opinionanted Digital Asset Management (DAM)
tool with minimal features. I wrote this so that I can quickly search my RAW
images on my slow access NAS and identify the images I want to bring over to my computer for processing.
Damage
scans the list of image directories provided and gathers theEXIF
information and writes them to a designatedmetadata
folder.- It replicates the directory structure of the image directories under
metadata
folder. For instance:- Consider
raw image folder
called/Volumes/Pictures/2000/Asia/03/22
- Assume that all the metadata is configured to be written under
/Volumes/Data/damage
- A directory called
/Volumes/Data/damage/Volumes/Pictures/2000/Asia/03/22
will be created and theEXIF
information will be written down with the name of the file - If the image is called
DSC001.NEF
, then theEXIF
file would be in/Volumes/Data/damage/Volumes/Pictures/2000/Asia/03/22/DSC001.NEF.json
- Consider
- A similar directory approach will be followed for generating thumbnails for images in a parallel directory structure
ElasticSearch
will be used to create search indices based on theEXIF
information- A simple server less Javascript GUI will be created which uses the index to render the images as thumbnails
- From the GUI, the following would be possible:
- Using
ElasticSearch
to create complex queries - Update
rating
,description
,title
,categories
andkeywords
. These will not be written back to the original RAW file. This will be added to the metadataJSON
file and indices will be updated. A utility may be provided later to copy this over to the original image, if needed - User can select the result of search and click a button to copy the selected files over to your local filesystem
- Using
- It won't update your original RAW files unless you ask for it
- It is not standards compliant with
XMP
and that support will not be added - This is not a polished product. It is a collection of scripts and a Javascript library. You have to run scripts by hand and open the JS in a browser
Assumes Mac OSX
- Install
exiftool
by following instructions in https://exiftool.org/ - Install
homebrew
https://brew.sh/ - Install
libmagic
bybrew install libmagic
- Install
ImageMagick
bybrew install pkg-config imagemagick ufraw
- Install
ElasticSearch
https://www.elastic.co/downloads/elasticsearch- Move the
elasticsearch
directory to/usr/local
and add/usr/local/elasticsearch-7.6.2/bin
to path - Start
elasticsearch
by running/usr/local/elasticsearch-7.6.2/bin/elasticsearch
. Figure out all the crappy Catalina security issues
- Move the
From the root directory, install gems by running bundle
. Enter sudo
password if asked for.
- Edit
config/constants.rb
and update the paths - Run
ruby prepare_directories.rb
. This creates the directory structure under#{METADATA_FOLDER}
and#{THUMBNAILS_FOLDER}
based on your#{RAW_FOLDERS}
and generates aimage_list.json
that contains all the images to scan
- Run
ruby download_metadata.rb
. This gathers all theEXIF
information for the images and writes them to an appropriate folder under metadata. This needs theimage_list.json
file generated in theprepare_directories
step
- Run
ruby create_thumbnails.rb
. This generates a 512 x 512 thumbnail for all your raw images and writes them under#{THUMBNAILS_FOLDER}
- Edit
config/elasticsearch.rb
if you want to change the number of shards or replicas or if you installedelasticsearch
in a non-standard port - Run
ruby prepare_index.rb
, just once, to create anelasticsearch
index calleddamage
- Run
ruby load_mapping.rb
to load the mappings for fields in the EXIF to fields in the index. Run this at least once after modifying the mappings
- Run
ruby create_index.rb
to load the metadata json files to theelasticsearch
index.
- If you got this far, you can start running queries against your elastic search index. Install a
elasticsearch
client likeelasticvue
for your browser and execute some queries.