Tue, 21 Jul 2009

Never, ever use an external USB hard drive as backup media

Filed under: Software and Technology by Hari
Posted at 17:44 IST (last updated: Tue, 21 Jul 2009 @ 21:35 IST)
This looks more like a public service announcement, but is actually a rant against external USB hard drives.

I was taking my usual backup of personal files and data when I noticed that I was unable to copy or delete files halfway through the process. After several frustrated attempts, I realized that the drive was emitted a faint, but perceptible clicking noise. Not an encouraging sign by any means. Even trusted and well-known brands (like Seagate) are bound to fail, as I learned today. :-(

Quite a lot of important data is stored on this hard disk and unfortunately there is no secondary backup for much of the data. As it is a fairly new hard disk bought just a few months ago, I hardly expected it to give me any problems. However, I learned a hard but valuable lesson today: never trust USB hard drives as a reliable backup option.

Even as I write this I'm trying my best to copy some of the data, but I've already encountered numerous I/O failures.

Luckily, in my case, most, if not all of these files can be retrieved in one form or another but will require a bit of work collecting the files from different locations. However, it has led me to think: what IS a safe backup medium in this day and age?

Let me evaluate some of the common options for offline backup (as opposed to data redundancy options like RAID):
  • Hard drives are bound to fail. Period. External ones seem especially susceptible to sudden failures. Internal hard drives aren't an option for archival purposes for obvious reasons.
    Bottom line: Hard disks are a very bad option, but are most convenient for storage of large files.
  • Optical media like CD-Rs and DVD+-Rs are notoriously susceptible to scratches and loss of reflectivity over a period of time. While optical media theoritically seems a safer bet than relying on a fragile magnetic disk, I've known them to fail on numerous occasions and had very little success in actually retrieving data from a bad optical medium.
    Bottom line: Reasonable solution, but inconvenient for bulky data running into several gigabytes and incremental backups are well-nigh impossible, requiring fresh media for every backup.
  • USB flash drives are known to fail and are, moreover aren't exactly large in size and cannot accommodate huge archives of audio and video files (as in my case).
    Bottom line: Not enough capacity and is not an archival quality option. Good in an emergency though.
  • Tape drives are practically unknown for home users. I'm not sure how many organizations continue to use tape archival for their backup purposes but I don't see it as a viable alternative for home users at any time.
    Bottom line: Inconvenient for home users, but seems to be practically the most reliable, tried and tested backup technology currently available in the market.
  • Other backup media are either specialized and are not easily accessible for desktop end users. Online backup is impractical even with high-speed internet connections and carries some obvious privacy and security risks, while local network backups require some kind of quality archival media already in place.
Any ideas on which backup device that a home user with considerable audio and video data (running to around 40-50 GB and moreover, not being downloaded music or movies, are far more valuable and irreplaceable) can use as a failsafe option under these conditions? I haven't a clue so far. Tape drives seem the most preferable solution, yet they are expensive and not easily available in the market, typically bulkier and require more physical storage space than hard drives or CD-ROM stacks for far less data capacity.

The real issue here is that hard drives are getting bigger in capacity and less reliable in terms of performance and stability. In my own experience, not one single brand or manufacturer inspires confidence for long-term reliability. If the reliability issue could be dealt with, there's no questioning the sheer convenience of a hard disk data store.

One would have thought that in this day and age, we could have devised at least one truly fail safe, reasonably inexpensive and convenient medium for permanent data archival.

20 comment(s)

Leave a comment »
  1. Have you considered the cloud?

    Shoot me for even suggesting it.

    My files are held together by prayer.

    Comment by Dion moult (visitor) on Tue, 21 Jul 2009 @ 21:19 IST #
  2. Have you considered the cloud?
    You mean backing up online on a cloud network? I'm not sure how it could help here. I'm not even sure I want to use a third party as a repository of my personal files, so storing such data online is not a possible solution. Besides transferring nearly 50 Gigs of data on a slow connection such as mine (averaging at a speed of 30-40 kB/s) would be excruciatingly slow and painful, let alone prone to errors and renders it practically impossible.

    Besides, what I really want is a solid offline backup mechanism which gives me peace of mind. :)

    My files are held together by prayer.
    That's the last and final strategy :biggrin:

    Comment by Hari (blog owner) on Tue, 21 Jul 2009 @ 21:27 IST #
  3. print the photos. paper works! we've still got papyrus! DVD's are bulky, but work with the proper care. Or build a raid server. (That's what we're planning on.)

    Comment by titanium (visitor) on Wed, 22 Jul 2009 @ 08:55 IST #
  4. Titanium, there are too many photos to print definitely more than 1000 and we couldn't afford that. Besides most of the critical data is tape converted MP3 audios and videos shot on a Sony Handycam. :)

    Right now I'm backing up the huge archive of audio data into DVD-Rs for peace of mind. That's the most important of the lot and represents literally hundreds of man hours of work of conversion from cassette tapes...

    Comment by Hari (blog owner) on Wed, 22 Jul 2009 @ 09:03 IST #
  5. DVDs are reasonably safe. Of course if you mishandle they will develop scratches and cracks. make 2 copies, use one for regualr use, and safely store the second one for back up of back up.

    Comment by Shrinidhi Hande (visitor) on Wed, 22 Jul 2009 @ 10:37 IST #
  6. DVDs are reasonably safe. Of course if you mishandle they will develop scratches and cracks. make 2 copies, use one for regualr use, and safely store the second one for back up of back up.
    The biggest disadvantage, as I see it is that DVDs are limited in storage and when you need 50-100 GB of data to be stored away safely, it becomes very difficult.

    Of course, DVD-Rs can always be used, but these days the laptop DVD drives (especially Slimtype drives) are more error-prone and not all DVDs tend to work on older systems. There are some issues with DVDs that render them less than useful for archival purposes.

    Good old CD-Rs are much more reliable and less error-prone but are far too small in capacity for audio and video data running to so many Gigabytes.

    Comment by Hari (blog owner) on Wed, 22 Jul 2009 @ 10:57 IST #
  7. Hari, we've all been suspect to failing hard drives. But the fact of the matter is, HDDs are still the most practical solution we have. I have seen enough HDDs blowing up and have some simple solutions:

    1. Work related documents: Keep them on a DVD and protect. Take backup whenever you need. Lose these, you're effed!

    2. Photos: You can of course upload the important ones to FlickR and stuff :)

    3. Music and movies: Share with your friends, so you always have a backup. My 110 GB music collection is in three places, so unless there's a ebomb going off somewhere, things are pretty safe :)

    4. Get a really good spikeguard. Trust me. It helps and prevents screw-ups in power fluctuation, which is the main cause of hard disks screwing up.

    5. Keep your HDD cool. Once I tried copying 500 GB of data and the room was warm. The HDD gor fried but thankfully was replaced under warranty. Under prolonged periods of copying, ensure it's under a fan on in an AC room.

    Comment by Chuck (visitor) on Wed, 22 Jul 2009 @ 12:40 IST #
  8. Thanks Chuck. Yes, I always use redundancy as a solution and it is a great life saver.

    However, there are many reasons why I don't give away my documents to friends or others and I feel more comfortable making private backups.

    Also I agree with you. Hard disk are the MOST convenient back up mechanism at present. However, exernal USB hard drives seem far more susceptible to mechanical and electrical problems than internal drives, which usually last quite long and work well. Also increasing capacity of drives seems to be a big issue.

    Heat can be a problem, but luckily I use a fan directly blowing at the computer, so it doesn't usually get so heated up as to cause problems.

    4. Get a really good spikeguard. Trust me. It helps and prevents screw-ups in power fluctuation, which is the main cause of hard disks screwing up.
    Actually it wouldn't have helped in my case. The external drive was self-powered and connected to my laptop, which is protected in any case by its own adaptor from overloads etc. In the main desktop system I use a UPS backup, so again, there's no real danger of spike problems.

    Comment by Hari (blog owner) on Wed, 22 Jul 2009 @ 13:00 IST #
  9. I've been thinking about backup recently as well. Offsite is a good choice, but how do you know a year from now, that website you backed up to will still be around? And if you're sending data so precious that you think you have to back it up, over the internet, should'nt it be encrypted?

    Comment by tjpike (visitor) on Fri, 24 Jul 2009 @ 07:44 IST #
  10. Good point, tjpike. That's why I am not a great fan of remote backup, especially when there's no chance that you will ever have physical access to the storage device.

    Comment by Hari (blog owner) on Fri, 24 Jul 2009 @ 08:26 IST #
  11. Hari, I've had too external drives fail on me. One was because I dropped it and the HD itself died and the other died due to the proprietary circuit board. The best external HD solution is using a plain external HD case (without the backup button or software embedded) and to do your own backups.

    I recently bought an Ovations RFID security case in which to house my old notebook HD. I haven't pulled it from the notebook yet, but I plan do so soon. Since it's designed as a backup solution, it doesn't have anything extra. The case is barely large enough to hold the 2 1/2 inch drive itself (one end of the case plugs directly into the drive). There is only the tiny, odd-shaped USB port like you find with camcorders and PSPs to contend with. I think it cost me $10.00.

    Anyway, the point is that you can use an external HD as a backup, but not the kind they offer right off the shelf with an HD included. Those do not last unless you rarely turn them on.

    Comment by RT Cunningham (visitor) on Sun, 26 Jul 2009 @ 20:59 IST #
  12. (I misspelled two as too and Ovation as Ovations)

    By the way, if your case clicks you can probably still use the drive. If it has a protected partition, you'll have to re-partition and reformat it. The circuit board in the case is usually at fault.

    Comment by RT Cunningham (visitor) on Sun, 26 Jul 2009 @ 21:03 IST #
  13. RT, thanks for the suggestion. It's a good idea to consider a professional backup-ready medium rather than an off-the-shelf external USB HDD.

    By the way, do self-powered USB drives fail more often? Mine was a self-powered USB HDD and didn't use any external power supply.

    As for drive partitioning, I did it myself. I actually used created ext3 partitions on the drive of around 147 GiB each (total actual capacity close to 297 GiB though advertised as "320 G";), because I use Linux.

    But the clicking noise definitely seems more of a real physical problem with the drive actually.

    Luckily redundancy has saved the day for me and nothing on that drive is lost.

    Comment by Hari (blog owner) on Sun, 26 Jul 2009 @ 21:37 IST #
  14. I had clicking in both instances. It could be either the drive or the drive controller. The only way to find out for sure is make it a temporary internal drive and test it.

    Both of mine had external power, so I don't think where the power comes from has anything to do with it.

    Comment by RT Cunningham (visitor) on Mon, 27 Jul 2009 @ 15:53 IST #
  15. RT, thanks for the tips. Currently I am wary of doing anything at all with this disk as it may invalidate the warranty.

    So I am going to give it back to the manufacturer as I will get a free replacement most likely as he said that he would replace it if he found any fault/defect. If he sees any sign that I have meddled physically with it, he will definitely not replace it.

    Also I will no longer use the USB hard disk as a prime backup. It will be a secondary back up.

    Comment by Hari (blog owner) on Mon, 27 Jul 2009 @ 16:48 IST #
  16. Hari,

    I dealt with a hard drive crash before. It was an internal 2.5" drive, though, and I ended up shelling out $700 to save 2 years of work and media: I didn't used to back up.

    I usually use an external drive for *backup* only -- that is, if my external drive crashes I still have all my data on my computer. I don't have enough media that I need an external drive.

    In your case why not use two disks in a Raid 5 or Raid 1 configuration? I think the price isn't that high nowadays and you can even scale up as your needs go up.

    Comment by Armchair Guy (visitor) on Sat, 1 Aug 2009 @ 23:05 IST #
  17. Armchair Guy, thanks for the suggestion. Actually I was also using the external drive for backup only.

    Since I am using a laptop primarily and cannot change the hardware, I am not sure how I can use a RAID 5 or RAID 1 configuration. If I ever use a desktop system again, I will definitely think of using RAID.

    Comment by Hari (blog owner) on Sun, 2 Aug 2009 @ 08:47 IST #
  18. Hari,

    I meant an external, USB-connected Raid enclosure. You get raid enclosures these days with up to 4 bays for hard drives.

    Of course a simple solution is to take an image of your first USB disk onto a second USB disk for extra protection...

    Comment by Armchair Guy (visitor) on Sun, 2 Aug 2009 @ 19:37 IST #
  19. I meant an external, USB-connected Raid enclosure. You get raid enclosures these days with up to 4 bays for hard drives.
    Ah, that explains it. I will have to enquire about the price of such a device.

    Of course a simple solution is to take an image of your first USB disk onto a second USB disk for extra protection...
    That's what I had actually. A second backup (in another internal hard disk though). Redundancy is the key, whether it's RAID-type setup or simply another device.

    Comment by Hari (blog owner) on Sun, 2 Aug 2009 @ 20:50 IST #
  20. Often users wonder how to backup files to a USB hard drive. This is an issue because some drives do not contain software to do the job. Microsoft released a tool called “ROBOCOPY” in Windows Vista, and 2008 Server that does a good job of synchronizing drives.
    For Pre-Vista Computers
    XCOPY use to be the tool of choice, but when encountering path and file names greater than 255 characters, XCOPY will generate an out of memory error and exit. If you are using a Windows XP or 2003 Server computer, ROBOCOPY can be downloaded from Microsoft in the “Windows Server 2003 Resource Kit Tools” located at:
    http://www.microsoft.com/download/en/details.aspx?id=17657
    Once you have downloaded and installed the resource kit tools, search for “ROBOCOPY*.*” and copy it to “C:\Windows\System32”. This will make the program available anywhere.
    Creating the Backup Script
    On the backup drive, we will create a one line batch file that contains the backup command. This batch file should be located on the backup drive. We want it here so that no matter what letter the drive is assigned, the backups will run.
    Using “Note Pad”, we will create a file called “Backup.BAT” and save it to your backup drive. The file should contain the following command:
    ROBOCOPY c:\ .\Backup\ /e /w:1 /r:1
    This will back up the whole hard drive to a folder called “Backup” where ever the “Backup.BAT” file is located: on the backup drive.
    How it Works
    ROBOCOPY will only copy the changed files, skipping a majority of the files that have not changed. If a file is in use or cannot be copied, ROBOCOPY will nicely skip the file and move on. This ensures that the backup will be completed to the best of the computer’s ability.
    If you have Outlook or QuickBooks on your computer, you will want to make sure that those programs are closed before doing a back. If you leave these programs open, the backup will skip their important data files.

    Shawn Zernik
    Internetwork Consulting

    Comment by shawn z (visitor) on Wed, 3 Aug 2011 @ 00:04 IST #

Leave a comment

First-time comments on this blog are moderated.
Your name*
Email ID*
(wont' be published)
Website
Your comments*
(No HTML allowed)		 :-) :-D :biggrin: :-P ;-) 8-) :-( :mad: |-| :oops: :-/ :-| :roll:
bold italic quote code
Code* captcha Enter the code you see in the image
* required fields