Larry Jordan’s Hard Drive Warning!
By S Simmons. Filed in Blogs and links, Internet resources, Mac software, from the net |
I was reading over Larry Jordan’s monthly newsletter today (and if you use Final Cut Pro and don’t subscribe then you should) and the second item under the contents is this: Hard Disk Warning! That’s always worth a second look so here is part of what Larry says:
Magnetic signals recorded on a hard disk are designed to be refreshed periodically. If your hard disks stay on, this happens automatically. However, if you store your projects to a removable hard drive, then store that hard drive on a shelf, unattached to a computer, those magnetic signals will fade over time… essentially, evaporating.
According to what I’ve been told, the life-span of a magnetic signal on a hard disk is between a year and a year and a half. The issue is complex, as you’ll see, but this is a MUCH shorter shelf-life than I was expecting.
Holy crap me too! I’m sure many of us keep hard drives around for entire projects as it’s much easier to plug in a drive and make changes than it is to recapture and rebuild an edit long after that edit is done. I’ve even found myself storing drives for clients after an edit not knowing it I will ever boot those drives up again. And I don’t always remember to pull them out and power them up as often as I should. This is even more important in the era of tapeless acquisition with XDCAM, P2 and especially RED. I can’t see into the future but I would bet that there are going to be a lot of pissed off RED camera owners out there in a 5 years when they go to boot up old hard drives and they won’t work. The lesson here is always archive your most important media to something other than hard drives and/or take time to check those drives on a regualar basis so they still work years from now. Just goes to show the need for an easy, reliable, cheap and (most importantly) accepted archiving system in the tapeless age. Larry has a lot more to say about this topic so click over and read the whole thing.
Wednesday, September 3rd 2008 at 3:07 pm
Wow… We’ve experienced the failure – Perhaps this is reason. Maybe paper punch cards aren’t so obsolete after all.
Wednesday, September 3rd 2008 at 3:29 pm
Ah, but with punch cards you have all that hanging chad confusion…
Someone (else) should write an app that automatically catalogs contents of hard drives on the shelf, includes an offline searchable database of the filenames/metadata in those files, and reminds you periodically to reconnect each drive in turn. As I understand it, accessing the entire disk is enough to “refresh” the drive and buy you another year on the shelf. This imaginary software of mine would do this automatically once the drive(s) are reconnected, and be sort of the central management portal of your hard disk archives.
And it’d be free/open source. Or really cheap.
Matt Jeppsen
FreshDV
Wednesday, September 3rd 2008 at 5:42 pm
yeah we just got an LTO-3A for just such a reason. mainly for RED stuff but also for general file backups for files that may not be on always powered drives all the time
Wednesday, September 3rd 2008 at 7:16 pm
Hey, I say punch cards for all! Until those holographic hard drives come along anyway!
Wednesday, September 3rd 2008 at 7:22 pm
I think realistically the lifespan of data on harddrives is probably longer than that, but that’s a safe number I reckon. I raised this issue about two-years ago at a P2 presentation with Barry Green – seeking then (and still) a good solution for inexpensive and simple data storage.
Optimally, for harddrive based storage the drives should be kept ‘online’ in an active state. In my case I’ve just built an OpenFiler NAS server for my soon-to-be-former workplace. Fairly inexpensive, relatively easy to maintain.
I’d be interested to know exactly what level of disk access is required to keep the drive ‘fresh’ – I’d have expected something like a defrag or similar block optimisation would be required.
Thursday, September 4th 2008 at 6:26 pm
I have an old Mac plus wiith an HD on it and I fired it up after 7 years and it ran fine.
Thursday, September 4th 2008 at 8:27 pm
What’s the brand on that 7 year old drive Mike? I think we all might want to get one!
Friday, September 5th 2008 at 12:47 am
I have had drives sitting in a box for years that work too – but don’t rely on it.
I also had a SCSI drive running 24/7 for just under 10 years as well. That was cool. Was fine until the power failed, and then it just didn’t spin up again… Hard drives are curious things.
Tuesday, September 9th 2008 at 10:34 am
What is the consensus on using DVD-R’s for solid state media backup? I have heard shelf-life figures from 2 years to 300 years. That’s quite a range!
http://digg.com/tech_news/CD-R_s_DVD-R_s_could_last_only_2_years
http://digitalcontentproducer.com/videoencodvd/revfeat/Archival_DVDs_Good_as_Gold090905/
Thursday, September 18th 2008 at 2:59 pm
@ Matt: We use CatDV as the “app that automatically catalogs contents of hard drives on the shelf, includes an offline searchable database of the filenames/metadata in those files”. Plug it in, it scans the files (they have a capture app for scanning tapes called Live Capture), creates previews (I make SD-res H.264 mp4s), and then metadata can be added. We have a whole team adding clips & info via their CatDV Server product.
FCP Server does this to a degree, but we’ve found CatDV to be more flexible. http://www.squarebox.co.uk/
As to the “remind me to refresh drives” bit, I could see that being an applescript or cron (once a year) notice. I suppose we could use CatDV’s “Play all clips as sequence” to play through every clip. Is that needed, or just a spin-up?
Monday, February 9th 2009 at 9:03 pm
In my experience as a magnetic media source-code librarian, PC tech, and general data-junkie, magnetic digital data can be expected to die within 5 years. This is because the signal is held on a little magnetic particle that is constantly vibrating in the magnetic field of the Earth and many other outside forces such as clocks, monitors, and house wiring, as well as the many other neighboring particles on the disk, which will have adjacent positive and negative orientations.
I am suspicious of the claim that data is automatically refreshed on a drive left powered up. I believe that to refresh at least most drives, the data must be read and re-written by an outside process such as imaging the drive and re-cloning it, or software such as SpinRite that refreshes by reading and writing. Just copying the files off and back onto the drive is good, but doesn’t refresh the basic partition, allocation tables, and directory structures.
Of course, I could be wrong.
Friday, May 1st 2009 at 4:40 am
That is the professional advice from Steve Gibson, the creator of Spinrite.
Spinrite reads and rewrites all the data on any drive or drive partition it operates on. Gibson highly encourages people to run Spinrite routinely, in order to refresh data that’s sitting dormant on a drive and rarely accessed. He claims the data fades over time unless it’s refreshed periodically.
Although I’m not totally sure that it happens, I suspect it does. The higher the frequency used for recording on magnetic material, the less it penetrates into the surface and the weaker the resulting magnetic orientation becomes. Modern drives already rely heavily on amplification to read the very weakly recorded data on the very thin film and it doesn’t take much to demagnetize the stored signal.
While I can’t see READING the data doing any good, defragging and relocating/rewriting the data certainly would refresh it. Older drives with lower density were “punched” a lot harder on recording than the modern ultra-high density drives. (Vertical recording -where the magnetic poles are set upright on the surface instead of laying end-to-end ….. probably helps the fade problem quite a bit, but still ….)
I was hoping to find some corroboration for Gibson’s claims from other drive specialists but have found nothing except for Larry Jordan’s article. I don’t especially trust Jordon’s word because I don’t believe there’s any evidence to back up his claim that drives refresh THEMSELVES during normal usage and/or that reading the data does anything to re-strengthen it.