Selection of media and storage locations for archiving and backup of photo archives

Any digital photographer has repeatedly encountered the situation when a flash drive with fresh images has already been inserted into the slot of a computer or reader, and there is no space on the working disk. It is unpleasant. It is even more unpleasant when two or three hours are killed for rewriting previous shots to optical disks, and then it turns out after all that some of the files on them are hopelessly corrupted. From the point of view of the photographer (and the art of photography itself), each shot is unique; losing it is equivalent to losing a moment of someone’s life, and for a photographer these moments can take place at a very sensitive time: after all, a picture needs to be taken, saved, processed, printed, shown ... I'm not talking about purely material losses, from collection to financial ones the death of any image is fraught for a commercial photographer.



The rational organization of a personal photo archive means a lot for the preservation of images in the home or a small photo studio. Important criteria for such an organization is regular archiving and backup (backup) of the images taken. On Habré already appeared a few years ago a good article explaining the fundamental difference between archiving and backup from the user's point of view. The essence is briefly: the archive is stored indefinitely and may be needed at any moment unknown in advance, while the backup is stored for some reasonable period of time and is needed only when the original data subjected to backup has been deleted or corrupted. then it was necessary to restore them.



You can articulate this difference in a different way. All data resulting from the work is archived, i.e. all but frankly dummy. Backup is done only for the data that is being worked directly at the moment.



Based on these criteria, we consider what properties archiving and backup should have when applied to a personal photo archive.

')

Archiving:



• Reliability of long-term storage (data should not be lost or deteriorated over time).

• Availability for cataloging (data can be found or sorted by criteria).

• Cheapness of storage (data should not take up much space, require a lot of care, trained personnel and other excessive overhead costs).



Backup (backup):



• Speed ​​- speed of copying and restoring data.

• High availability of data for rapid recovery.

• Reliability of short-term or medium-term storage (as opposed to long-term archival storage).

• The reasonableness of the overhead of the organization of the backup.



As the owner of a small photo studio, I had the opportunity to meet with the experience of organizing both archiving and data backup, not only the photos themselves, but also the documents accompanying these images, from studio content to commercial contracts. I want to tell you more about my experience in this matter below.





Fig. 1. The organization of archiving and backup. The blue arrows are normal data flow (save and retrieve). Orange arrows - a data stream in case of a critical situation (recovery) requiring special attention.

Archiving

We start with archiving, as an older operation in nature, since photographers kept their images in archives in various ways in the pre-digital era. Archive, in my opinion, you need all the photos, from the banal "I'm at a corporate party" or "my girlfriend in Montmartre" to the works filled with the deepest author's meaning. A technically unsuccessful picture may contain a beautiful plot or a unique instant; what seemed blurred or blurry will probably become in the future a substrate for a collage. My advice: archive and store absolutely everything!



Unfortunately, in the era of cameras with 20-40 megapixels on the sensor, the size of a digital image grows completely disproportionately to its quality. My Nikon D800 gives you a compressed in-camera JPEG image with an average size of 16 MB per image, while one RAW file reaches an epic 76 megabytes! A DVD optical disc, a typical archiving device in the early digital age, will now contain no more than a hundred or two hundred RAW files from a modern camera. Meanwhile, the number of photos in one photo shoot can reach several thousand shots! (Basically, I think this is wrong. If you shoot more than 200-300 frames per day, you either bomb or click thoughtlessly, hoping that at least one out of a thousand frames will be successful. Unfortunately, the discussion of this topic goes beyond the scope of the conversation about archiving.)



Therefore, the media for storing the photo archive should first of all have a rather large volume. For my studio, I use Western Digital My Book Thunderbolt Edition dual-drive external drives configured as a RAID 1 array. A few years ago, of course, I used a DVD, and I think I got off cheaply, losing more than 4% of files with this storage method. Optical discs are, to the extreme, unreliable, mainly due to the structural imperfection of their drives, which easily damage the media when reading or writing speed; I consider the fact that many photographers continue to use them as a result of a misunderstanding.



Solid state drives (flash memory or SSD) are also excellent carriers; The speed of their work is higher than that of traditional disks, and reliability increases due to resistance to accidental magnetization and the absence of mechanical parts. Unfortunately, these devices are still quite expensive. A wonderful, but very expensive archiving option - storing captured memory cards as they fill them up, as if they were photographs. This option still seems uneconomical, but in reality, comparing it with the cost of a film frame, you cannot consider it such a terrible expense, especially if you are an old school photographer and do not beat it off at random, but build each frame intelligently and tastefully.



Recently, an alternative to local storage in the form of relatively cheap network-based cloud services with low latency (data access speed) has appeared. These services include Amazon Glacier. Unfortunately, the cost of a cloud account is still too high, and software to automate the exchange with cloud services is not so much, but breakthroughs are outlined in this direction. Subsequently, cloud services can be an attractive alternative or, rather, an addition to storing archives in the office or at home.





Table 1. Comparison of various types of data carriers on operational properties.



I am skeptical about the long-term storage of photo archives on network drives (NAS) or FTP servers, since with this method of storage the archive is exposed to all typical network hazards: hacker attacks, viruses, equipment failure, or a third-party error can destroy the fruits of the photographer’s many years of work. In any case, the network archive should not be the only place in which your photos exist! Keep copies off-line, at least for your best work.



Since the archive has a tendency to grow rapidly, I strongly advise you to have special software for cataloging and automating work with the archive, as well as to perform backup of the entire archive before making any significant changes to its work (for example, transferring it to a new storage location). Even the cost of licensed software for archiving and storage is negligible compared to the cost of the cost of photographic equipment.



It is worth to mention that many photographers who are confronted with the dangers of the digital age prefer to keep their “masterpieces” exclusively in the form of paper prints. This approach seems to me to the extreme impractical, since the paper and the emulsion are not less threatened, and the range of possibilities is significantly narrowed. Therefore, speaking of the archiving of digital photographs, in our time it is necessary to keep in mind the electronic storage of the archive.



A separate method for storing photos is in photo banks and on photo sites. This method allows you to store images not as abstract data sets, but as ready-made cards that can be viewed. The volume of accounts on most specialized sites is already sufficient for putting there very large photo collections. However, this principle has some limitations - on archiving and storing commercial photos, RAW files, on uploading automation, etc. Therefore, I still do not consider this method of organizing photo collections as a serious alternative to traditional archiving.



You can archive manually, but it is even better to have special software that creates and checks your archive on a certain regular basis. In the simplest case, these can be command language scripts or batch files that copy the contents of your directories to the right place.



Making copies of a photo session on optical disks or solid-state memory upon the customer's request is not, in fact, a form of archiving, therefore, what was said above does not apply to it.

Backup (backup)

The need for backup exists from the minute when the photographer presses the button to take a picture and until the file is sent to the archive and / or to print. Even manufacturers of digital cameras are aware of the importance of backup. Most of the current professional and top amateur models are equipped with two slots for memory cards, one of which can be configured to automatically back up the other. The trade offers a lot of gadgets, like digital photo albums and mini-disks, for backing up the captured data from the card in the field.



Backup is extremely important in digital data processing, especially when working with files in JPEG format. As you know, files of this format deteriorate in quality with each re-saving, therefore the number of operations with them in graphic editors should be minimized. If, however, you still zoropoli JPEG file, you will need its fresh original version to start all over again.



Backup is an integral part of the proper organization of the home photo archive (I told about it in another article ). The main task is to choose the right media for storing backups. Recently, I am inclined to this option when using two media: a hard disk that is directly connected to the computer (DAS - Direct-Attached Storage) and an online storage (NAS or cloud service). The first of these options has the reliability and speed in its merits, the second - the readiness of the stored data and the ability to physically share their storage location with the location of the computer.



Files are backed up to the connected device (full backup). The backup execution tasks are started strictly manually. If the device is permanently connected (which is convenient from the point of view of work, but not always desirable for security reasons), then it makes sense to configure automatic backups of working directories for it (full - once a week, differential - daily).



Backup to a network drive or FTP is performed as planned. Once a week, a full backup of the working directory (Workflow) is performed, while daily data from the directory is copied to the archive incrementally or, if the possibilities permit, differential (differences between incremental and differential backup are a separate topic).



To store, in my opinion, it makes sense to 3 weekly copies. In everyday practice, there is rarely a situation where the use of current working data is delayed for a month or longer. After this period, files can be sent to the archive.



Since the current working data after backup takes up significantly less space than the entire photo archive, you can use commercial cloud services to work with it, for example, Amazon S3 and Dropbox based on it. This is not so cheap, but it provides access to your data from anywhere in the world where there is a stable Internet. If you are a reporter, travel photographer or just travel around the world a lot, this opportunity can be extremely useful. In addition, in most cases, using cloud storage you can easily transfer the copied information to another person or transfer it to another computer.



An interesting alternative to the above use of cloud services may be to configure personal clouds on network-attached storage media (NAS). Some types of backup software, like most network storage vendors, offer this solution.



Backup, unlike archiving, is very inconvenient to carry out manually. You either forget to complete the task, or copy not everything, or make another unpleasant mistake at the most inappropriate moment. There are many different software, built-in, free and commercial, designed for backup (I use the program Handy Backup ).

Conclusion

It turns out that archiving and backup require several different approaches to the choice of media. The main common criterion is the reasonable price for their use, including overhead costs; in all other respects, the requirements for data carriers for archives and the current backup are almost the opposite.



The relatively low price of local and network hard drives makes them nowadays the main type of media for organizing personal photo archives and backup working data. On the other hand, the ever-increasing manufacturability and cheapening of the cost of cloud services can gradually change this situation in the near future, making photo storage “in the clouds” the preferred way to organize the archive.



I will briefly list the conclusions I came to when studying for myself the question of storing photo data and its backup copies.



• Optical discs are outdated morally and physically.

• Local drives are good and cheap, but can deteriorate or be erased.

• Network storage is also good, but there is a danger of catching a virus.

• Cloud services are convenient to almost everyone, but still too expensive.

• However, the future is in cloud services.

• Now it makes sense to combine backup and archiving on local disks and on cloud services (including private clouds, raised on the basis of their own NAS).

• Investments in backup software are justified if the software can work with all the above types of storage media.

• Always back up memory cards removed from the camera. Is always!



Basically, that's it. I wish you to easily enter the history of photography with your successful and well-preserved photographs, or at least please them for several generations of your home ones!