Tape Drives a Viable Alternative to DVDRs?

[Boeman]

I currently use brand name DVDRs to backup my data but have found that while the cost is low, the reliability is simply unacceptable. A few scratches is seemingly all it takes to render a DVD (and its accompanying data) null.

I've had to replace my DVDr drive once already after burning 100 DVDs (at which point all I was getting were coasters).

Of course, there's the inconvenience of managing large numbers of DVDs as a whole that I could certainly live without. So far I'm totaling 130 or so DVDs.

I am now considering a tape drive for archival purposes. Despite being vintage 70's technology, I have never utilized tape drives before so please consider me uninitiated.

My needs are as follows:

* I use OSX 10.4 (Panther).

* I'm primarily looking for a storage solution for high resolution photographs and captured digital video files (files pertaining to graphic design and video editing).

* I won't be doing any rotational backups or storing files temporarily on a regular basis, rather I'll be adding data as I go along.

* I would like to be able to see and modify the file structure on the tape just as I would a regular hard disk as seen in the Finder. I may need to relocate items into a different folder, replace files with updated versions or delete files and folders altogether. I won't be erasing tapes very often.

* Speed is not a serious issue. I'm more interested in capacity (50GB+) and reliability (tapes that won't corrupt or damage data that is written on to it as frequently as DVDrs).

* I would like an interface other than SCSI (I have no SCSI adapter).

* My budget is around $400 CND.

From the limited information I've been able to gather via Google, it appears that apart from being prohibitively expensive, tape drives are an all or nothing proposition; when you store information, even a small amount, you replace all the data on the tape. Is this true?

Any advice from fellow forum members would be most appreciated.

[Michael Emrys]

I've never used a tape drive, so take the following with a helping of salt. Since the drives do exist and are on the market, presumably there are some people who find them a viable solution to their backup needs. On the other hand, you've already mentioned the cost, and it seems to me that the continuing costs would also be high. Vid files would take a lot of tape, I should think (but I might be mistaken on that point).

I'm trying to understand how your DVDs get so scratched as to be unusable. Normal scratching, even if it causes a reading problem, can be buffed out and the disk restored. If you are handling and storing them with even a modicum of care, they should seldom get scratched at all.

Michael

[JonS]

All* commercial backup is done to tape. Tape is - or at least used to be - the cheapest form of removable media, on a cents/gigabyte basis. They look expensive because each tape is so large. They are also simple and reliable.

Downside is the start up costs, and the slow access times if/when you need to get anything off the tape.

* to a first order approximation

[Soddball]

Have I just entered a discussion from 1996? Has nobody heard of USB hard drives? Cheap, fast, reliable. If you're worried about one failing, buy two.

[Subvet]

I would stay away from tape. What about just using hard drives? Mirror a couple of large hard drives. I know you said your aren't that worried about speed, but I'm not sure you appreciate just how slow tape is. Hard drives are HUNDREDS of times faster than tape. Tapes wear out too. You'll have to replace them every couple of years if you want the reliability to be there. To top it off, hard drives are cheaper.

I use a tape backup solution every day at my job. We are definitely looking at converting over to a backup server using hard drives in the near future.

[Wilhammer]

I shall agree on USB Hard Drives; the BEST solution.

Stay away from the minis based on laptop drives; go for something meaty based on Western Digital, like a Laci EHDD.

[Lars]

I've used tape drives and as noted, slow. Very slow. CD's in all their flavors,become another management hassle.

Just get another hard drive, in whatever flavor you wish, and backup to that. Shouldn't run you more than $50.

[Wicky]

External firewire drive i.e Lacie Quadra

Or if there is space in your bay (if you have a tower mac) you could simply add another HD or two

I've have a additional 250GB internal SATA drive for important data - then have everything backed up via firewire with OSX10.5's TimeMachine on an external 1tb Lacie.

Plus SMARTReporter to keep an eye open for any imminent SATA HD failure

If you really want to absolutely fail safe with your data then think about setting up a RAID array

[Redwolf]

Oh my. Memories.

DAT tapes were what we labeled "the first write-only device in computer science" and Exabyte drives were investment hell since the drives broke too often. Which would have been fine if one drive could have reliably read another drive's tapes.

But there's no good way around it.

Harddrive base backup such as using a USB or firewire drive suck, because

• 
  
• you overwrite your old backup. If your primary data storage dies during the backup you have nothing
  
• no incremental backup. If you silently corrupt a file you won't notice until long after all backups already have the corrupted version
  
• harddrives often die after being stored on a shelf for a while
  

Myself I when through a lot of trouble with a whole server in the basement. That server has a raid-5 (one disk can die without data loss), filesystem snapshots (so that I can go back through some increments) and runs 24/7 to detect dying harddrives. Not exactly a perfect solution either.

I have heard DLT tapes being held in much better esteem than the 1990ties junk, but tapes are slow, in particular if you use incremental backup.

[Soddball]

Oh my. Memories.

Harddrive base backup such as using a USB or firewire drive suck, because

• 
  
• you overwrite your old backup. If your primary data storage dies during the backup you have nothing
  
• no incremental backup. If you silently corrupt a file you won't notice until long after all backups already have the corrupted version
  
• harddrives often die after being stored on a shelf for a while
  

'Hard drives often die after being stored on a shelf for a while' is pure BS. Often? How often? 70%? A while? How long? Six months? That phrase is deliberately couched in vague terms to frighten rather than inform. Hard drives do fail, but not 'often' and not after being stored on a shelf.

For $50 you can buy yourself a piece of backup software that'll do incremental. For $100 you can buy two 500GB hard drives, and you can swap them fortnightly for your backups. Then you can stick them in a fireproof safe on another site. That's about as secure as you can reasonably make your backup.

[Wilhammer]

Indeed. I have a score, easily, of old HDDs, converted to Storage via USB using a SATA/IDE to USB converter.

With all the HDD I have ever seen fail, they failed early (manufacture defect) (<5%, were dropped (20%), or got over heated (laptops mostly) (75%).

In an Enterprise, I've seen a few SCSI drives fail after thousands of hours of work, and most I can reasonably trace to a sudden power event.

[Lars]

And even when they fail, you can still get the data recovered.

[Wilhammer]

Yes, very easy to do, btw.

Fire up a PC with a plugged in USB drive using a Boot CD (Ubuntu works great for this, or UBCD4Win) and you can recover much if not all your data.

For the thermally warped, a trip to the freezer often helps....

That Score of HDDs I have - all are 'failed' - which means just a few sectors are failed - and a Complete Reformatting will result in the MBR/FAT being 'told' which sectors are bad, and they won't be used. These are 'shock' failure drives.

Some are Catastrophic - as the drive continues to be used, the number of sectors getting hosed increases.

For messier HDD recoveries, many tools exist - I like GetDataBack myself. It has worked for me many times, even getting stuff of of Bad Sectors.

[RSColonel_131st]

I run an automatic tape loader as primary means of backup and archival needs here in the company, and I sorted out the backup for my friend's digital animation studio with a different system, so I think I can add some qualified opinion here.

First, what I can't quite read out of your posting Boeman, is if you are looking for backup (i.e., when your computer crashes and fails, you want to get your data back) or if you are looking to store old data for later (in some cases, much later) reference and reuse. You say "archival" but that usually means NOT being able to change file structure, folders or update files afterwards - so you seem in fact to require a mix of backup and archival.

Both are different systems - for backup, you usually just keep the latest version of each file - for archive, you want all the version history of each file.

I'm also not clear on how much data you're talking here. 130DVDs are either close to 600 gigs, or 1100 gigs of data. Is that all "current" data you need to access daily and which changes daily, or does that include old data which is stored for potential reuse later?

So, on general terms, without knowing those details:

My company has a volume of about 300gig total data, most of which doesn't change daily or even monthly, but each of my tapes has a volume of up to 600 gig, so we just do a full backup daily, and put the last tape of each month into an external storage safe somewhere else. That means I can replace a damaged file instantly, or find a file that had been changed or deleted 18 months ago.

Problem for you is that tape is relatively expensive for the drives, and does not scale well with less space needed - 50GB per unit is on the low side, but the drive alone would cost you more than your 400CND, and you need some clever software as well to get the best out of it. It also does not do most of the things you asked, like easily changing files and moving them around.

For your budget and situation (as well as I can make it out) what I would recommend is one step above the suggested external USB-Drives - get an ethernet-connected NAS (Network Attached Storage) system. That is a small stand-alone tower case with a network adapter, which runs 4 or more disks in a RAID configuration, allowing you to lose and replace one hard drive out of the 4 without loss of data or real downtime. Depending on the size of the drives, you can get up to 1500GB storage which is basically safe against loss trough mechanical failure of the disks (though a power surge or spike may still take out more than one drive, so get an Uninteruptable Power Supply to go with it). Problem with RAID, compared to just copying files onto multiple disks, is that if the RAID controller hardware fails, you are often out of luck if you can't get the same hardware, since a different controller will not be able to put togehter those pieces of data spread out on 4 disks. But that is a low concern with the right brand name stuff.

This NAS would be your primary storage where you can directly work from, over network share. Bonus is that more than one system can access those files easily from all over your home network.

Still, for long-term storage/archival needs, I would see to get a Blue-Ray Burner when they come down a bit in price, and then write to BR. That's 50 gigs of data per disk, pretty safe long term (though of course tape and DVD has been in use for longer, so more reliable statistics exsist on storage life of those media).

Ideally you want to keep the files you regulary need on the NAS for quick access and moving them around/changing them is not a problem; when you're definitly done with a project and run out of space on the NAS, put those files on an archival disk.

That's how I'd do it - but just for the NAS with over a terrabyte you might need a bit more than 400CND. For that budget, all you can get are indeed two USB disks to copy files on daily or weekly in a rotation.

[Redwolf]

'Hard drives often die after being stored on a shelf for a while' is pure BS. Often? How often? 70%? A while? How long? Six months? That phrase is deliberately couched in vague terms to frighten rather than inform. Hard drives do fail, but not 'often' and not after being stored on a shelf.

I've never seen any statistics about this, but I have experienced it myself just this year, and there have been series of harddrives that were particularly vulnerable to this (due to some chemical that would turn into glue on non-use). Since at the time where you use your backup harddrive you cannot yet know whether you are affected.

For $50 you can buy yourself a piece of backup software that'll do incremental. For $100 you can buy two 500GB hard drives, and you can swap them fortnightly for your backups. Then you can stick them in a fireproof safe on another site. That's about as secure as you can reasonably make your backup.

That still leaves you with the problem of silently coorrupted files. If you detect any corruption later than 2 days after the damage you have overwritten both harddrives with the corrupted version.

The advantage of the tape solution is, or rather should be, that the medium is supposed to be so cheap that you can have old copies around, say one tape per month into permanent storage.

[Boeman]

Thank you for the responses.

RSColonel_131st, I work mostly with motion graphics/3D animation as a side job and as a hobby.

I shoot both uncompressed DV and still camera images as part of my pipeline. Lately, as I've had more time during the fall, I have amassed close to 15 gigs per week of footage and imagery; some of them requiring hours of travel.

I tend not to use the captured data as is, but rather take various segments that are edited as part of a larger composition. Often times I will save the edits for later use (thus requiring more space). I'll usually compress files that won't see immediate usage when archived.

The original unedited footage is always saved since certain projects require different segments or manipulations of video/stills. Without them, I would have to re-visit the site and conduct another re-shoot.

Essentially, they are not unlike stock assets available to me on demand should I need them. I suppose I would not classify them as super critical mission data - that would be the project files themselves. The stock files take up the lion's share of space, though.

From what I've gathered thus far, tape is not the best solution for me given the above scenario?

[Wilhammer]

Sounds like you need the NAS storage solution outlined above, as well as secondary backup.

[Redwolf]

You should also consider a raid system directly in your computer, if you think your computer is reasonably safe from corruption and you are not in the habit of accidentally deleting or overwriting your files.

The raid as the primary storage in the workstation means that you don't lose a day worth of data, which you would if you have a non-raid primary and a raid or non-raid nightly backup on a harddrive.

[RSColonel_131st]

Boeman, now I have a more complete picture. What you are doing is similar to what my friend does in his company, albeit he only has about 700 gigs worth of stored data after a few years. 15 gig a week is a heavy collection - if you keep up this speed.

In this case, I would reverse my above scenario. Your primary backup and archive would ideally be with a Blu Ray writer. Put the original data on there as soon as captured, then add edits later when you are sure they are done. This will be "permanent" so no changes afterwards, just as archival storage for your "stock collection".

For the data currently in editing and transforming, you can then either use a smaller NAS (4x 160GB drives would give you 480GB of RAID5 storage with the ability to lose one drive at no risk). If the NAS is too expensive, do as Redwolf says and get two 500GB drives mirrored in your system (either trough software, or a motherboard able to do SATA RAID 1). Not as reliable as a separate NAS (since a big hardware crash on the system is likely to fry your data) but you can also one drive in this config.

Normally I would recommend a large "working storage" like a NAS, and smaller archive media to put away old projects, but in your case it sounds like you want the raw initial data and any edits on permanent archive, and just the current working files on a harddrive backup.

Blu-Ray Writers are not THAT expensive anymore (I saw some here for about 300CND) and if you're really short on cash, just add an external USB drive for daily backup instead of a RAID or NAS.

[__Yossarian0815[jby]]

re: harddrive failures:

google published their experience with HDs a few months ago. The bottom line (IIRC): harddrives fail at the beginning of their lifetime or very late. ambient conditions (heat) and frequency of use are totally irrelevant.

I would use RAID1 with at least 3 drives and backup on an external HD (USB, firewire or external SATA)

forget tapes

[Redwolf]

Wait, I didn't recommend going raid-1/mirroring with that onboard software raid junk. You either use decent software raid such as the one in Linux or you need a hardware controller. The onboard junk has nuked a lot of people's data when they relied on working raid-1 functionality.

A reboot during a recovery while using this stuff can easily lead to nuked data as experience has shows that the required split between BIOS and main driver sometimes leads the BIOS, when rebooting during a half-done recovery, writes the empty drives data over the good drive and similar gags.

That onboard raid gesocks is really only good for raid-0.

You don't need an expensive hardware controller. Even one that uses the main CPU for raid5 XORing is fine these days because the CPUs are fast enough. But you want all software involved during boot and under the OS to be written by a reputable RAID controller vendor. Not by the morons at NVidia (I don't think I have to explain that on this forum).