4:4:4 12-bit Uncompressed DVX100

[Ben Syverson]

A further problem with SATA II is that, looking at the spec, I see no mention of peer-to-peer connectivity. One of the basic principles of Firewire is that you can use it to hook two Firewire devices together without a computer. For example, you could hook a camera directly up to another camera, or a camera to a hard drive. This capability was expanded with FW800.

With SATA II, it seems that you'd either have to build some very sophisticated electronics, or rely on a computer to sit between the DVX100 and your drive. This is clearly non-optimal.

Reading the spec, it seems that the main design goal of SATA II is pretty similar to SATA: to provide an ultra-fast channel of data direct from a disk or disks (they mention RAID configurations often) to a computer.

So SATA II is never going to replace the USB or Firewire families as a convenient general-purpose interface.

And I'd think twice before I discounted FW800. Final Cut Pro already supports DVCPRO-HD over Firewire, and HDV utilizes Firewire as well. If you want to be doing HD work in the next few years, you'll still be using your old pal FW. That'll help keep the entire FW market charging forward...

- ben

[Ben Syverson]

"..SATA II is a final spec..."

Huh, that's interesting. Because the SerialATA website ( http://www.serialata.org/ ) calls the draft a "Release Candidate" undergoing a 30 day review as of 4/22/04.

I'm looking forward to buying some SATA II gear later this summer! I'll plug it right into my SATA II port, which is located right next to the Unicorn dock and all six FW3200 ports. We may see SATA II gear in 6 months, but its main relevance will be internal storage and high-end external RAID solutions. When they start putting SATA II ports on laptops, then we can talk.

Clearly we do disagree, and it's not just about the marketshare. But tell me exactly what you searched for in Google. How on earth did you find numbers for hard drive marketshare within the video/media industry? That's what I want to see, since we're talking about uncompressed DVX100 lest we forget, which is a niche issue.

I think Juan should rely on his own experience and intelligence, which has led him straight to FW800, an interface which is fast, readily available, and apparently not a problem to implement. I don't understand why we're still talking about SATA II and "wireless USB" and unicorns...

- ben

[Stephen van Vuuren]

I did not save all my google links, but search for "sata" "serial ata" etc...

SATA is peer-to-peer based, though I don't expect it to see it in DV or HDV cams as it is way overkill. But for HD and faster cameras (like the Viper, Arri), the 3 Gb/sec of SATA II might be better than akward SCSI implementations especially now that Seagate has got Native Command Queuing supported.

I had forgotten the name but a google search for serial ata peer to peer gave this link for the chipset company that already has SATA II controllers ready:

https://www.marvell.com/products/sto...a/88SX60xx.jsp

The built in SATA power connection standard is also something I like.

[Peter Plevritis]

I'm for the quickest, least complex and most direct solution.

A FW800 port connected to an external FW800 drive - Juan's original idea - sounds great.

It'd be nice to also be able to connect it to a FW800 equiped PC/Mac as to record to a large internal RAID array with lots of storage. Of course that means you are tethered to a computer but hey sometimes that's okay. Plus I think a firewire cable can run a long way.

I don't think it should have an internal drive. Mostly because it will probably delay getting the thing out and about. Real world production will get the proper feedback for a MkII. What, already a MkII being discussed!

A video output would be very useful for previewing and SDI output would be fantastic. Of course keeping with my no-major-added-delays theme.

If I had to add something...
When I hit record on the camera - the tape and the drive start to record. Maybe there is a way to sense that switch-over. But that might involving soldering and I am weary of that.

But really a separate record button is fine. Please add a record light though.

[Ben Syverson]

As far as I know, The Viper and Arri are utilizing HD-SDI. The Arri outputs just about exactly 3Gb/sec in normal mode, which is cutting it kind of close for a single SATA II connection. I think it may go over 3Gb/sec in 75fps mode. They're running the Arri over 3 HD-SDI's, each of which have 1.5Gb/sec...

Of course, maybe SATA II will come into play for these high end systems, but... so what? Weren't you the one who brought up marketshare? These systems have barely seen the light of day, and are strictly rental material. Whatever interface they're using on those giants has no bearing on anyone here...

My view of the storage market may be incomplete as a whole, but I have a pretty good sense of what professionals and prosumers are using for video. When I make guesses about marketshare, they're simply based on what I see people using, and aren't intended to be taken as empirically or statistically sound... So the next time you declare accusatorially that someone doesn't know the "actual sales numbers" and hint vaguely at Google results from god-knows-where, you might want to be acquainted with said numbers yourself.

Firewire is built into the computers and DV equipment we use now, and the HD equipment that we'll be using -- HDV and DVCPRO-HD. For that reason alone, Firewire will continue to be a massive force in the video world, and is the best forward-looking choice for a portable direct-to-disk box.

SATA II is going to be awesome for renderfarms and workstations, but until I can look at my camera and my laptop and see two matching SATA II ports, we're still living in a FW world.

- ben

[Stephen van Vuuren]

The google results I was hinting at are proprietary studies from Instat and iSuppli if you must know. They are only available for purchase, not for posting and linking. But the results can be inferred form the discussions and new articles on technology news sites and abstracts that reference them.

Firewire 800 was barely mentioned in the context.

First of all, you keep using "firewire" and "SATA" as equivalents - they are not. SATA is both a drive technology and a interface. Firewire is an interface only.

Firewire is a great interface and the standard for DV interconnects. I use it every day.

But storage is another matter. Capturing tapes to NLE drives is an enormous waste of time and money. Direct aquistion to disk is the future. The best low cost disk technology is SATA - now and in the future - not firewire; as firewire is not a disk technology but an interface stacked on top of disk technology.

Some firewire drives today and all firewire drives in the future will have SATA drives inside.

I'm suggested to Juan that he start with a SATA drive and firewire interface to begin with. So we agree here since you have not indicated what type of drive Juan should put inside the firewire enclosure.

But a SATA II external port will be faster, more efficient and eventually lower cost due to volume. And they will be available soon - I'm not wrong about that.

[Peter Plevritis]

Some numbers.

-image size: 773x495
-3 channel 16bit TIFF file = 48bits
-single image size (uncompressed) = 2.189 MB
-52.536 MB/s at 24 fps
-60 minutes at 24fps = 185.74 GB

As has been discussed 16bit tiffs are way more than needed for the 12(10 now) bit data. But it does make it much easier to work with.


Some other numbers to note.

D1 (601) 422 ntsc uncompressed video:
720x486, 0.67 MB/frame, 20 MB/s, 60min=70.4GB

HD1080 444 24fps uncompressed video:
1920x1080, 5.9 MB/frame, 142.4 MB/s, 60min=692.5GB

[Jason Rodriguez]

SATA isn't the problem that I used to think it was. There are external enclosures (although not powered which might be a problem for a hand-held camera), and the interface does support hot-swapping. Additionally you can purchase cardbus adapters for SATA, so there shouldn't be any problems on the laptop front. Frankly I think it's only a matter of time before SATA does overtake the external hard-drive market over firewire800, but that may take a year or more before stuff is readily available, right now SATA stuff is quite sparse whereas firewire800 is everywhere. So while we may argue over this fact over and over, I'd say we go with the solution that works now and gives us greater choice, because frankly if we wait around for another year just to get a hard-drive interface that we want, then they'll be something else on the horizon and nothing will ever get done.

BTW, on the other topic of 10-bit linear Cineons, all those files should be compatable with every program that supports Cineon/DPX-the reason being that the lin-log conversion takes place via a look-up table inside Shake, combustion, etc. All these programs read the log file as a linear file, that's why they look so "light" when you open them. After applying a LUT, you get the linear image the way it should look. With Juan's approach of using a linear Cineon file, you'll simply ignore the loading of the LUT.

One thing though, I thought the RGB files off the A/D converters was 12-bit. If you use a 10-bit linear Cineon, then you're loosing a lot of the highlight dynamic range. I'd say use either a 10-bit LOG Cineon to perserve that dynamic range, or else use a 16-bit Tiff file. I would find it very unfortunate that you'd have all this dynamic range and then clip off the top two bits to squeeze it into a 10-bit linear file, destroying the advantages of the extreme highlights that this camera can handle. Just my .02 cents.

[Juan P. Pertierra]

I agree with Jason, and this is the same reason i came to integrate FW800 to the design. As much as SATA looks great and right now seems like it is the future, FW800 is what is available right now. Perhaps when I apply this mod to one of the new 3-CCD HD cameras, SATA will be more mainstream and we can switch over to that....besides, the added bandwidth WILL be needed then. :)

The cineon format supports 12-bit linear, so it can still accomodate all the information. I think someone said that 10-bit is the most common, but like jason says, i'd rather go 16-bit TIFF than sacrifice 2-bits/channel of color, unless the user explicitly wants to.

Juan

[Jason Rodriguez]

BTW, just curious, do the 7200RPM Hitachi 2.5" HD's sustain a high enough data rate to support the 16-bit TIFF's? I was thinking you might be able to use one of those smaller drives for a "magazine" if you will, and then offload that onto a much cheaper 3.5" HD attached to a laptop after the fact (or just keep swapping out the 2.5" drives if you have the cash :-)

[Juan P. Pertierra]

I checked out all the small laptop drives (including the hitachi's) that i could find, and it didn't seem like they could handle it, at least by any significant margin.

However, if you have anything particular in mind, just see if it can do at least 42MB/sec sustained write....the numbers peter posted are correct, yet a bit over-bandwidth for the actual application, since it actually captures 12-bitsRGB(36bits) and then later converts the data to 16-bit TIFF's. So the actual sustained write rate is ~42Mb/sec.

This number is for the final application however, when dummy bits are stripped out. Right now, because of all the dummy bits present in the raw images, my test computer is recording ~63MB/sec sustained, with a hard disk 2 years old that i bought at circuit city. Stripping out the dummy data and using a modern FW800 drive is a much better situation. :)

Juan

[Stephen van Vuuren]

Juan:

For some reason I'm having a really hard time making this point. There is no such thing as a "FW 800" drive. Inside, you will have to put either a EIDE or SATA drive.

You are absolutely correct Stephen. I have not seen any FW800 external cases that accept SATA drives, only EIDE. I don't think it wouldn't make a difference in speed whether a SATA or EIDE drive was used in a FW800 external case anyway.

[Juan P. Pertierra]

Stephen:

I agree, and i know exactly what you mean. :)

But, what should we call my LaCie 200GB drive from now on? =)

Juan

[Ben Syverson]

<< For some reason I'm having a really hard time making this point. >>

It's because it's not a very good one. We're not discussing the type of actual hard drive mechanism to use, because this project does not include an internal drive. We are discussing what kind of port to put on the box. You will then connect a cable to the port to an external drive. The external drive has an internal drive in it -- and yes, if we're talking about a FW800 drive, 99.99% of the time, there's an IDE drive in there.

But you seem to be a bit confused as to what you're arguing for. Are you really saying we should put an SATA port on Juan's box? That's what you seemed to be saying a few posts ago.

<<There is no such thing as a "FW 800" drive.>>

Wow. Actually, the term "external drive" refers to the whole package, just like "computer" doesn't just refer to the motherboard. "External drive" is often shortened to "drive," if you've already made it clear that it's external. Therefore, it's perfectly reasonable to call an external drive a "FW800 drive," or a "Firewire Drive," or a "USB2 drive," etc. It doesn't matter whether there's a SCSI or EIDE or SATA II drive. If the back of the durn thing has Firewire ports, it's a Firewire drive. Unless you're on some kind of Quixotic quest to abolish abbreviations, I'll keep on saying "firewire drive" rather than "EIDE hard drive within a FW800 enclosure."

<< inside, you will have to put either a EIDE or SATA drive. >>

Or a SCSI drive. So what are you saying? That we should use SATA in FW800 enclosures?

<< The google results I was hinting at are proprietary studies from Instat and iSuppli >>

Convenient! But unless these were studies specifically of the media/video market, and unless they were conducted at least 6 months after FW800 came out, I refuse to believe that you're somehow more informed than me on this particular issue.

Bottom line: I know FW800 is the best choice, Juan knows, Peter knows, Jason knows, everyone seems to agree that FW800 is the best currently available choice... except for you, Stephen...

- ben