Evernote data protection: how we get rid of broken disks

In our post “Three rules for data protection in Evernote,” we already talked about some of the measures we are taking to protect our data so that our users can trust our service. In fact, there are many more such measures, and today I would like to tell you about one important point: what do we do when hard drives fail.

You probably had to read stories about people who bought computers from their hands and found there various information from the previous owner, with very confidential data. Therefore, Evernote takes decommissioning drives very seriously.

We use both hard drives and solid state drives (SSDs) in our user data storage infrastructure. Hard drives are mechanical in nature, and therefore, like all things with moving parts, break down sooner or later. The SSD has a different failure pattern: such drives have a limited number of rewriting cycles, after which they become read-only.

We provide redundant storage with hardware-based RAID controllers. This means that the failure of one disk does not affect the safety of your data. In addition, we take preventive measures to identify disks that may fail by tracking data transfer errors and forecasts provided by the drive itself . If these values ​​reach a critical threshold, we replace the disk, without waiting for a breakdown, usually on the same or the next working day. Sometimes the disks just break without warning, and then our task is to replace them as quickly as possible.
')
As a result, we still have all these broken disks, which may contain user data. The ATA instruction set contains the Secure Erase feature, which overwrites each track on a disk, making data recovery almost impossible. All this is great, but for this you need to drive in working condition. And in our case, most of the failed drives no longer function, so this feature does not suit us.

Disks are expensive and, as a rule, come with a guarantee (usually three years, sometimes even more). Manufacturers typically require customers to return non-performing discs for replacement under the warranty program. But since our disks may contain user data, and we are not able to use the very function of Secure Erase, we cannot afford to send disks for repair or replacement and thereby risk user data. Fortunately for such cases, most manufacturers offer special replacement programs, known as “Black Hole” (Black Hole). Specific conditions may vary from one company to another, but usually it’s enough for a customer to send a disc faceplate and some form or a written statement about the physical destruction of a disk.

In general, our approach to working with broken disks is to destroy them, where we also adhere to the principle of redundancy.

There is such a National Institute of Standards and Technology ( NIST ) - a US government agency whose tasks include the development and publication of technology manuals for other US agencies. These guides are available online for free and generally meet industry standards. The NIST publication number 800-88 ( “Guidelines for cleaning media” ) covers both physical and electronic recording forms. The approach to Evernote is based on this instruction.

Our work with broken disks consists of the following steps:

• Disks sent for destruction are stored in a safe place.
• The front panel of the disks is removed (see photo # 1, # 2 and # 3), for which you need several different types of screwdrivers (what would we do without iFixit sets !).
  
• The disc is placed in a Garner Products HD-2 degausser (photo # 4), where data is safely erased.
  
• Then the media is physically destroyed using a device that physically crushes the disk with a powerful wedge ( Garner Products PD-4 in Photo # 5 and # 6). This makes the disc completely unusable (Photo # 7 and # 8).
  
  
  
• Broken parts of the disk are then sent for recycling.
• The faceplates (photo # 9) are sent to the respective manufacturers, and the disks that arrived for replacement come into operation again.
  

The goal of all these operations is to make sure that the safety of user data is never at risk in the process of working with disks. This principle, combined with strict adherence to NIST guidelines and other industry standards, ensures that we use proven and reliable methods.