Cataloging and reserving personal photo archives

In our age of publicly available mirror and mirrorless cameras with a matrix resolution of 16 megapixels and higher, a home photo archive can easily become the place where, as they say, the devil breaks his leg. Due to the size of the files stored in it, the photo archive can easily clog not only a laptop disk, but also a medium-sized NAS or a multi-disk external drive. Meanwhile, for a photographer, if he is not only “bombed,” almost every photograph taken has value; it will seem now that this unsharp thing is suitable only for a basket, but no - in a couple of years it will turn out that a good collage can come out of this picture! Nostalgic memories brought in digital format from France, Egypt, from the last bell, from the wedding of a cousin to the first mother-in-law in the village of Maliye Dosy and so on - I don’t mention at all. This passes under the category of "store forever."

Files multiply, clog screw by screw, are lost in the abyss of time and the same names, until in the end the data remains merged into a uniform common mess, in which you can not find anything sensible, and besides, half of the necessary files are worn or defective.

Therefore, cataloging a personal photo library is necessary, as is its regular backup (backup). There are a number of utilities that offer this feature, usually with photo viewing features. However, for most users, such utilities are not very much in demand. In fact, there are two circles of photographers who would fully have enough functionality offered by iPhoto, Aperture or Picasa; these are either amateurs who shoot in JPEG and don’t bother with the processing, or professionals with a single camera / system purchased for centuries, in which the process of working with photo files is pre-verified and thought through to the last movement. The rest are faced with problems, the range of which I will try to briefly outline below.
')

• A lot of files with pictures. One shooting day is easily able to bring a photographer from 500 to 2000 shots. Naturally, some of them will be immediately eliminated, but everything else needs to be processed, and the processed one should be safely stored!
• Duplicate file names. You will definitely have more than once a situation when you copy a directory with the name “100xxxxx”, the files in which start with “DSCN0001” into a folder where there is already such a directory, and such file names. What happens when it happens depends entirely on your attentiveness. In the worst case, you may lose the results of one or more shootings!
• Different processing algorithms for multiple cameras. In many cases, you have to look for the optimal converter for RAW files shot by a particular camera, since using the same program is not justified by the quality of the output data. As an example, you can use Adobe Lightroom, which, according to many, is insufficiently revealing the color possibilities and the sharpness of a number of modern models. In such cases, alternative developers are used, including native software cameras, RPP , Raw Therapee or Capture One. This eliminates or makes it difficult to use a standard single developer as a cataloger or software for making backups.
• Storage errors. With a typical intensity of data flow associated with the processing of a variety of photos, the wear and tear of working computer equipment increases, up to the breakdown of working disks. If at this time there was still a photo collection not re-stored somewhere on the disk, this would lead to irreparable losses.

The above problems, as already noted, are solved using two template approaches:

• Proper organization of photo archives contributes to the preservation, retrieval and use of data in the required mode.
• Backup (backup) and archiving eliminates the loss of captured and processed photos.

Next, I will briefly describe my personal experience in organizing an amateur photo archive and its supporting infrastructure.

Organization of photo archives working catalogs

No backup utilities will save the situation if there is trash and confusion in the archive itself. Therefore, the first action necessary to preserve the photo archive in order is correct rubrication and cataloging.

Most digital cameras (a pleasant exception - Pentax mirror models, for example) write pictures in folders with uninformative names, like “100_FUJI” or “103NIKON”. When I copy a folder from a card or camera to the working directory of the photo archive, I do not regret half a minute to rename it to the following format: [CAMERA_THE IMAGED_ WHEN REMOVED]. For example, at the output, the folder name may turn from an obscure “330_FUJI” into an absolutely meaningful “FUJI-X20_Corporative_Now_2010”. To lose such information, you see, is much more difficult.

By the way, the folder for copying work files on all computers is called “Workflow”.

The same applies to the names of the files themselves. My files are divided into raw and processed in editors. If the files of the first type can be called something like “1010394.NEF”, then I always rename the processed files according to the format: [THIS IMAGE_DXB], where D is the length and B is the height of the final file in pixels. Let's say the final name may look like "Bridge_through_Reku_Kvay_900h675.jpg". Thus, I always know well enough whether I processed a certain file or not. The processed files are stored in the Workflow directory, in the same folders where the original images were located. By necessity, I move these files to special folders entitled “To Print” and “To Publish”, which are stored in the same Workflow base directory.

From time to time I manually archive (but not backup!) The contents of the Workflow folder. In this case, the folders with which I ended up messing around are copied to another place (to another physical disk!) With the same name that was given to them after copying from the camera. These folders are stored in directories named [Archive_YEAR] - say, “Archive_2013”.

Thus, I always know what and where is in my photo archive.

Backup, archiving and data security planning

First, I’ll clarify in brief the difference between backup and archiving. Archiving is storing data in a safe place. But backup is storing data in a safe place with regular updates and the ability to quickly recover.

For archiving electronic data, including photos, it’s enough just to copy them somewhere. Backup implies more complex data handling: regular scheduled copying, checking integrity, data integrity, convenience and accuracy of recovery if necessary.

For a full backup, copying to the right place is probably not enough. It makes sense to use specialized software: either something with built-in functions, like iPhoto on Mac OS X or the popular 7-Zip archiver , or specialized backup programs like EASEUS ToDo Backup , Handy Backup or Symantec Backup Exec.

Yes, I know, most people will not understand me. Programs for backup are usually licensed, and at your home Linux. Well, bash allows you to write all the necessary backup scripts yourself, and native open source Linux products, like Bacula or rsync, are not at all offended by the functionality. What to use here is the choice of taste rather than principle. I personally believe that after buying a DSLR for 3000 bucks and corresponding to the price of interchangeable optics, saving only 50-100 dollars on licensed software can force only a very basic attitude to life. But, I repeat, I’m entirely for open-source, and I don’t want to underestimate the functionality of free software! Let a hundred flowers bloom, as long as they all work properly with our photo archives.

Photo Library Backup Tools

So, in addition to archiving the work done, I have the need to regularly back up Workflow directories on all computers, and also from time to time to make sure that I have fresh backup copies of archive files and folders made in this and previous years. I would prefer to copy the Workflow catalog daily, but archives, as already mentioned, it makes sense to touch once a month if they are fresh, and only once every six months or year to overwrite the old archive data if necessary.
It is worth mentioning here that I have three working machines: one (stationary system unit) works under Windows 7, another (laptop computer for field work) under Fedora Core, and the third (laptop for representative needs and comfortable work with multimedia) - under Mac OS X latest version.

I’ll start, perhaps, with Macintosh, as the most popular system among foreign photographers. The standard backup tool on Mac OS X - Time Machine - seemed to me frankly awful. Either I couldn’t figure out how to work with it, or I had the wrong instruction, but it turned out that Time Machine just takes a daily (or more often) whole copy of your Macintosh hard drive, and rewrites it in a dedicated place. And so - until the place runs out. Then the oldest copies on the selected media are erased, and the whole process continues with a new one ... Probably, this is an excellent means of ensuring security against system errors and failures, but it’s impossible to talk seriously about backing up working directories with such methods. In my opinion, this approach is too wasteful.

Offhand, from the fluently read reviews, I never picked up a single tool for organizing a target backup under Mac OS X. Several applications, including Get Backup, did not satisfy any of my requirements as a user. You just have to copy the folder manually to your home NAS.

Linux (in my case, Fedora Core) is much richer with adequate backup applications. Of these, we should first mention Bacula - a good, universal tool for organizing backup. In addition, in the case of Linux, I regularly use the above-mentioned bash scripts to automate the copy process. These scripts are so elementary that I don’t see much point in bringing them here.

Finally, backup utilities for Windows are a great many. For a long time I used the built-in backup function of Windows, until I accidentally came across the Russian software Handy Backup (this is such a program for backups of the whole row, with the antediluvian interface, but with good functionality). True, she does not know how to work with CD / DVD, but, taking into account the above, for me now it is not particularly critical.

Scheduling data preservation tasks looks like this. Every day, each of the Workflow catalogs is copied (incrementally) onto a personal cloud raised on the basis of the Western Digital My Book Live Duo NAS. Once a week, a full backup of the Workflow catalogs is made, while the old versions are stored for the previous 3 weeks, and then deleted. The archive is located on another NAS, Apple Time Capsule, where the data from Workflow catalogs are differentially moved once a week. From time to time the contents of the catalogs are completely copied to the archive manually. As for the archive itself, from time to time it is also fully downloaded to the external Western Digital My Book Duo disk, which is irregularly connected to the computer via USB. I used to perform this operation manually, but now I use the capabilities of Handy Backup, where a special task was created for this purpose.

I move the processed files to the “To Print” and “To Publish” directories of the Workflow folder on a PC (workstation), where I run another task at least once a week through the backup program, this time - the task of writing to USB flash drives. Flash drives are placed in paper boxes with a date and an approximate list of contents. This operation is performed separately from the recording of disks or USB-carriers with final results for storing photo collections or transferring them to customers.

Conclusion

The described photo archive organization has been providing me with reliability and trouble-free access to any selection of photos for several years. During this time, I changed several cameras, purchased new computer equipment, but did not lose a single shot. Therefore, I take the liberty of recommending the simple method of storing and backing up photos that I have described as an alternative to the “proprietary” catalogers.