SDHC long term storage reliability

[David Heath]

Originally Posted by Simon Wyndham

But like I said, c'mon people, what is the likelyhood you will actually need *your* video in even 10 years time, let alone 25, 50 or 100!?

Coincidentally, after not looking at these forums for a few days, exactly this point came up yesterday with some friends we visited over Christmas. They (like many other people) bought a video camera when their first daughter was born, and now that she's coming up to 21 had the thought of looking at the videos again. That was when they realised that the tapes were S-VHS-C, all they had as a player was a VHS machine, and the camera's long gone.

I’ve ended up promising that if they can find the C adaptor, I can dub them from S-VHS to DVD, but the point is that when they bought the camera in the first place and spent a lot of time using it, they just assumed that there would be no problem in viewing the material in years to come – no more than opening an old photo album.

OK, if I hadn’t got the facilities, there are businesses which can do the conversions at present – but it would be at quite a cost, and that cost is likely to be more as materials get older and the replay equipment gets scarcer. I’d bet a lot that a great many AVC-HD cameras get sold every day for people to film such as babys first steps or words, and I’d equally bet that the people who buy them just assume that the images will be accessible whenever wanted in the future.

This all assumes the recordings themselves are fine, and it's just a question of getting the right hardware. I've recently looked at quite a few tapes from around that period, and so far, no problem. But what about newer technologies?

Originally Posted by Simon Wyndham

The other thing that is favourable is that, assuming I understand things right, SD cards do not need to keep a voltage in order to hold data. So things like voltage leaks over time leading to loss of data shouldn't be a problem.

The best link I’ve found about flash memory principles is: Flash memory - Wikipedia, the free encyclopedia and the basic principle is that data is stored by the presence or absence of electrons on a floating gate, the state being read by how their electric field influences a transistor gate. That article says little about how long data will remain uncorrupted, other than:

Because the FG (floating gate) is electrically isolated by its insulating layer, any electrons placed on it are trapped there and, under normal conditions, will not discharge for many years.

So the big question becomes just what “many” translates to? I’d be pretty sure the data would be OK in a years time, and expect there not to be a problem in 5 or 10 years based on experience, but what about 25 years, 50 even?

Originally Posted by Peter Kraft

Sandisk talks about a lifespan of 100 years. Ok.

Do you have a reference for that, Peter? I ask not just through academic interest, a client is trying to put a proposal together for a project where archivability is probably the most important single factor. The problem being faced is that any proposal has to be backed up with sound, referenced evidence. So a Sandisk salesman saying “oh, you can expect them to last 100 years!” is no use, a Sandisk White Paper saying the same thing with evidence would be hugely useful.

The other thing I’ve learnt to be careful of is the difference between physical lifespan and data lifespan. If you talk about a device having a lifespan of 100 years, it’s conceivable that you may still be able to write and read data to/from it in 100 years time, but any given set of data may only last a lot shorter period of time.

[Simon Wyndham]

I’ve ended up promising that if they can find the C adaptor, I can dub them from S-VHS to DVD, but the point is that when they bought the camera in the first place and spent a lot of time using it, they just assumed that there would be no problem in viewing the material in years to come – no more than opening an old photo album.

The biggest difference here though is that the information that have is stored on a linear storage device in analogue form. Whereas with digital files it will always be easy to transfer information as new technology comes along. Also the way in which new versions of Firewire and USB retain backwards compatibility means that for a long time yet information could be retrieved from such devices and cards.

So the big question becomes just what “many” translates to? I’d be pretty sure the data would be OK in a years time, and expect there not to be a problem in 5 or 10 years based on experience, but what about 25 years, 50 even?

Yes, there is that question, which is why if the data really is important that established industry proved archive solutions should be used such as DLT and LTO. This doesn't help the average family, but I think that is just a case of being vigilant, making more than one copy, and checking things regularly.

So a Sandisk salesman saying

I'd imagine they have a white paper somewhere. The 100 year thing was in their press launch for the product.

[Greg Boston]

Originally Posted by Simon Wyndham

You make the analogy of 5.25" diskss. Well I was using them and I managed to copy my data to 3.5" disks and then eventually hard disks.

Simon, you young whippersnapper, I remember using 8" floppy disks. And believe me, at that diameter, they were pretty floppy. (hehe)

Ok seriously, I subscribe to the same ideas about archiving. Just remember to keep transferring your digital media to newer forms of storage. I think the biggest issue facing us in the future is miniaturization getting absurd. It wouldn't take much to lose the micro-SD cards we already have available!

-gb-

[Simon Wyndham]

Yes, the size is getting silly, and came up in a conversation the other day. Size is something that is as important as robustness in a production environment. It would be great if someone could manufacture an SDHC card in an Express card body for this very reason.

[Greg Boston]

Originally Posted by Simon Wyndham

Yes, the size is getting silly, and came up in a conversation the other day. Size is something that is as important as robustness in a production environment. It would be great if someone could manufacture an SDHC card in an Express card body for this very reason.

As you well know, that's exactly what P2 is. Albeit with a bit more circuitry to handle the RAID functionality. 4 SD memory devices inside although I doubt they are in their usual plastic enclosure. Which brings up another interesting memory of mine...

A few years back, there was an MP3 player on the market using a CF Microdrive for storage. When word got out among digital photographers, people were cannibalizing the players for the memory drive because it was cheaper to buy the whole player than the CF Microdrive card in stand alone form.

-gb-

[David Heath]

Originally Posted by Simon Wyndham

Whereas with digital files it will always be easy to transfer information as new technology comes along.

Agreed in principle, but it is assuming that even with the right hardware, the original files are still readable. A bit of degradation on an analogue tape recording may mean a bit more hiss, or a few seconds missing. A bit of degradation on a digital file may mean "corrupt file, cannot be read"!

.......why if the data really is important that established industry proved archive solutions should be used such as DLT and LTO. This doesn't help the average family, but I think that is just a case of being vigilant, making more than one copy, and checking things regularly.

For a large broadcast organisation, then yes, DLT or LTO and active management, but as you say, not much help to an average family. Sometimes the best family memories are the ones that are come across by accident, after lying undisturbed for years. Unfortunate if it's a photo album where some of the pictures are a bit faded, but better that than a totally unreadable digital media?

The advice about making a copy and checking regularly is sound, but how many families do you think get told it when they purchase their AVC-HD SD camcorder? And how much does it say in the instruction books about media longevity? That's why it would be so valuable to have even an idea of the lifetime of the data on a standard flash card.

I'd imagine they have a white paper somewhere. The 100 year thing was in their press launch for the product.

I've found the Sandisk white papers section, and can't find anything about this topic. Anything said about reliability seems solely to concern short and medium terms.

I took the 100 year figure to refer to normal flash memory, but now understand it refers to the newer write-once cards - SanDisk | Business Products | Flash Memory Cards | SanDisk 3D One-Time-Programmable (OTP) Memory – and not data on standard flash cards. If the OTP cards work as some are hoping, then they will indeed be close to the Holy Grail for many in the industry. To quote from a page linked to by Sandisk (SanDisk | Business Products | Flash Memory Cards | SanDisk 3D One-Time-Programmable (OTP) Memory ) “To program the memory, a permanent physical change is made to the structure, as opposed to storing charge, eliminating reliability concerns from charge leakage and data corruption. To program the memory, a permanent physical change is made to the structure, as opposed to storing charge, eliminating reliability concerns from charge leakage and data corruption.”

But the assumption seems to be that one of these cards will be able to be placed in a camera like the EX, and a standard (but unalterable) recording made. I’m not sure this is the case, since although Sandisk talk about compatability with existing flash devices, that may be from the point of view of reading from them, all that Sandisk say is:

SanDisk 3D-OTP Memory has been designed for electrical and mechanical compatibility with standard (NAND) flash devices and can, therefore, work interchangeably with flash card formats and fit into millions of existing slots without requiring any hardware redesign.

When they talk about writing to the memory, they seem to imply that specialized technology is necessary:

SanDisk offers a full, Quick-Turn Programming Service that meets the critical cost and schedule requirements of content providers, designers, and OEMs.

I’d love to be proven wrong, and I suppose the best thing to do is write directly to Sandisk, but I strongly suspect that 3D-OTP is intended more at markets like memory for portable GPS devices, to store such as mapping data on. Come the upgrade, you just get a new OTP SDHC card and throw the old one away.

As far as video use, they could also be very useful as a Blu-Ray alternative.......