How do I get 4:4:4 1080p 12bit unaltered, uncompressed into my computer cheap?? at DVinfo.net

Adam Koppe · June 21st, 2004, 08:51 PM

Ok I have been doing lots of reading on these threads and others and it seems a few people are trying to get an answer to this question

What is the cheapest way to get uncompressed, unaltered, 4:4:4 10 or 12bit 1920x1080 30fps onto a hard drive for manipulation by FCP HD on a Mac or whatever windows program for PC.

I am interested in getting a Mac solution

From what I have gathered

[size=+1.5]1. Storage[/size]

Seems from my rudimentary calculations

1920pixles x 1080pixles x 36bit(colour res) x 30fps
= 279Mbytes/sec

An 8 drive SATA raid with 250MB Maxtor MaXLine III’s will yield 38.2MBx8 = 305.6MB/s (at it’s slowest inner tracks) which is 1.8hrs of this uncompressed video.

X-Raid does this with an external enclosure and via Fibre Channel (400MB/s) to a card in the PCI-X in the Mac. There seems to be other similar cheaper non-apple options as well. SATA-II seems to do this cheaper again when it comes.
So IMHO storage is do-able. Larger faster HD arrays can be built also

[size=+1.5]2.Camera[/size]

This is the biggy, what camera will output uncompressed, unaltered 4:4:4 10 or 12bit 1920x1080 30fps data (with usable lenses for shooting, to preserve resolution, cheap is good) to a computer or straight to the hard drive array.

Cameras that I have seen discussions of

Summix
Imperx
Silicon Imaging
Redlake
Kinetta (but this seems to far away)
Sony X300 (seems more expensive than the rest)

Issues

Connection to comp or HDD:
HD-SDI...............too slow
Dual HD-SDI...............Fast enough
Firewire 800...............Too Slow
Cameralink...............Fast enough
Gigabit Ethernet...............too slow

Drivers for the Mac???

Interface card for the Mac????
Aja's KONA 2 Dual HD-SDI
Decklink HD
Can you connect straight to HDD Array???

Format
A Format than can retain the info and be read or converted to a readable form (without info loss) for FCP HD

CCD’s

Larger CCD better???

3 better than 1????

Is there a camera that can do this, if there are multiple which one gives a better picture, f-stops, lenses etc.

Suggestions for a setup is what I am looking for

David Newman · June 21st, 2004, 10:28 PM

If you are considering any of the single chip Bayer solutions your data rate is a third of what you have calculated. If you intend to pre-process Bayer data there is little advantage to producing 4:4:4 data as you really only have an approximation closer to 4:2:0. Split the difference and use 4:2:2 like most of the industry then HD-SDI will work perfectly for 1920x1080 4:2:2 10bit @ 30fps.

Rob Lohman · June 22nd, 2004, 04:40 AM

I'm not sure where you are getting at with this. What you want
is simply not possible. Since you want it cheap. This is my humble
opinion on the matter.

The datarates are way too large for anything to handle that is
CHEAP. But you say we have cheap RAID controllers these days.
Yes we do. The only problem is that the normal PCI bus cannot
handle these datarates. So everything needs to be PCI 66
which futhers adds to cost.

Then I'm not sure how you would edit this (in what format).

Keep in mind that camera's from Silicon Imaging cannot record
to harddisks etc. As you probably know we are working on such
a system now, but it will do 1280x720 10 bit to begin with.

I've said it before and I will say it again. I'm not sure why
everyone is so obsessed with HD and 10/12 bit recording. Would
it be nice to have, definitely. But a lot of people do not seem to
have a clue as to what it entails as far as:

1) costs (camera + interfaces + software + storage + computer + monitor etc. etc.)
2) portability
3) ease of use (ie there is no camera white balance, digital effects etc.)
4) BACKUP! (people already have problems with DV backups!)
5) editing problems (size, support etc.)
6) extra time and costs on your set (need monitor and probably other support systems in place as well)
7) lenses (research, buying etc.)
8) power on set

Then again I'm seriously doubtfull on how many people will
actually shell out the amount of cash (think at least $4000 for
the full system) for a homebuilt system with basically NO
SUPPORT or WARRANTY (if you build your own!). Yes we have
a community, but that's a different thing.

The system will probably be even more expensive if someone where
to build and sell. This due to manufacturing, research and support
costs.

And this is only for 10 bit 1280x720 I'm talking about now!

I'm not trying to scare anyone away, but I would like to add a
few realistic (in my mind) pointers to all of this. Let's start fairly
(and I don't say this with ease, but it ain't easy!) easy and see
where this and the market takes us.

Don't forget to write a good story first.... to shoot with this magic cam

I will also see where this thread takes us because we already
have 3-4 (!) threads running on this subject. It get's very hard to
follow what is going on with all these threads, especially for
outside companies trying to support this all. I understand the
need for discussing certain problems in more detail, but in my
mind we just are a pretty long way from discussing the things
proposed at the beginning of this thread.

Chris Hurd · June 22nd, 2004, 05:50 AM

I'm not sure that I understand the need for capturing uncompressed, unaltered, 4:4:4 twelve-bit 1080i video. How are the current HD formats (such as DVCPro HD) inadequate? How can you actually see any difference. Compression is not a bad word.

Steve Nordhauser · June 22nd, 2004, 06:28 AM

Chris:
I agree, there is nothing inherently wrong with compression. It carries two evil threats that are not necessary but always lurking. The first is loss of real information that will never be recovered. That is why it is universally agreed that lossless compression is acceptable (barring the other evil), "virtually lossless" (hard to define other than with respect to the image noise floor) is probably fine, and somewhat lossy may be OK for the final release but not for the raw recording.

The second 'bite your butt' is processing time. Obviously, you need to record fast enough not to drop a frame. Even with buffering, the average rate out to disk must be able to meet the average in from the camera. Fine. Compression helps that. But now, instead of data going by DMA (direct memory access) to a disk controller, it goes to system memory, the processor does the compression (this is why simple is better in real time) back out to memory and written to the HD in blocks.

Some people are doing this, depending on the camera clock rate, bit depth, etc. Cineform and others seem to have pretty high clock rates they can handle. I'm sure you need a faster CPU but maybe only a single fast HD - important for the removable pack people.

Chris Hurd · June 22nd, 2004, 06:44 AM

Thanks, Steve -- it's great to have you here, by the way.

Rob Lohman · June 22nd, 2004, 06:52 AM

Chris: As Steve excellently explained it is a tradeoff, isn't it always.

I can clearly see DV compression on more complex scenes. So
that's a bit too much. We are definitely working on getting some
form of compression (lossless) in the system. The problem we
have (as some points pointed out to by Steve):

1) we want as less amount of harddisks as possible. This requires compression

2) we have a limited amount of resources available to do this compression since we want to go smaller in the future (less CPU speed) and it is much data we need to handle

3) a lot of compression algorithms / codecs can't be afforded by us to license like some of the higher ends camera's can do (or develop themselves)

4) 99% of the algorithms out there are for RGB or YUV data at mostly 8 bit (that's probably 75% or so). We operate in Bayer.

In regards to Bayer. This actually helps us (not in quality, but
in data bandwidth needed). We get a 1.5:1 reduction already
coming from the camera. Then if we store it efficiently we get
a reduction of 1.8:1.

This is without any quality loss (the signal from the camera is
stored as it is coming in, only stored more efficient) and no
compression. So with compression we do hope to bring this
down even futher. Althought we can't make it lossy due to it
being Bayer. The Bayer format needs to be kept intact to do
the best Bayer to RGB/YUV conversion we can do. This process
will be done as a form of "capture" stage on the computer.
Since there is both time and more processing power available
there.

As Steve indicated we can't use DMA. So it is all a tradeoff...

Laurence Maher · June 23rd, 2004, 05:46 AM

My personal take on the matter is this:

FCP HD does at best 4:2:2 1080p (virtually) uncompressed via SDI. So really, you don't need a camera or storage capable of doing any more than that if you're working with FCP HD. I am just about to get a G5 with FCP HD with the later intention of getting one of these upcoming silicon imagin cameras (1080p due out within 12 weeks) and using it with some form of SDI or camera link card. I'm looking at recording a feature at 4:2:2 1080p around 200 Mbps, which is far from uncompressed, but is much better than the CineAlta or Varicam. Personally, I think this is the sweet spot for cost and quality. Going lower will get you a flick that looks too scarily close to "video", and going higher will break your bank book. As long as I can fool most people and have decent latitude, I'm okay. The rest can probably be made up for via the way you shoot it (ND filters, etc.).

I do agree that 1080 4:2:2 is close to a necessary evil, but then again, if we can get 720p at 4:4:4, that's pretty much gonna compete with it, if not be better.

Rob Lohman · June 23rd, 2004, 07:45 AM

Well, these CMOS chips basically do 4:2:2 already. Or that's how
you can look at it. It is NOT 4:4:4 unless what everyone is writing.

So those datarates are still a lot.

One thing I was wondering. I thought HD was 720p or 1080i
(notice the interlaced inidicator there!), according to standards.

Anyone know for sure?

Rob Scott · June 23rd, 2004, 08:33 AM

Quote:

Rob Lohman wrote:
I thought HD was 720p or 1080i

That's correct. See http://support.gateway.com/s/CsmrElt...984faq42.shtml for all 18 (!) formats.

Note that there are 1080p options - 24 fps, for example.

Jim Lafferty · June 23rd, 2004, 12:33 PM

Quote:

So really, you don't need a camera or storage capable of doing any more than that if you're working with FCP HD. I am just about to get a G5 with FCP HD with the later intention of getting one of these upcoming silicon imagin cameras

You're not thinking of dragging the G5 around during production, are you?

- jim

Valeriu Campan · June 23rd, 2004, 04:03 PM

A Viper camera control panel with its HDs are looking like 10 G5 put together. Not to mention the 21" monitors ASO...

Laurence Maher · June 24th, 2004, 12:36 AM

If I have to do that to get my film shot, you bet I'll drag it (VERY CAREFULLY!) But others are talking about putting together a cheap PC that I might use for capturing, then transfer the files to the mac. We'll see. Any way you look at it, it's very true that real filmmaking (especially theater-quality HD) is extemely bulky. The idea here is not to get around the amount of work necessary for a great film. The idea is the cost difference. HD will end up being much cheaper with these new camera systems and computer storage than messing with the costs of film. (Well, if you do it right).

June 21st, 2004, 08:51 PM	#1
Adam Koppe Tourist Join Date: Jun 2004 Posts: 1	How do I get 4:4:4 1080p 12bit unaltered, uncompressed into my computer cheap?? Ok I have been doing lots of reading on these threads and others and it seems a few people are trying to get an answer to this question What is the cheapest way to get uncompressed, unaltered, 4:4:4 10 or 12bit 1920x1080 30fps onto a hard drive for manipulation by FCP HD on a Mac or whatever windows program for PC. I am interested in getting a Mac solution From what I have gathered [size=+1.5]1. Storage[/size] Seems from my rudimentary calculations 1920pixles x 1080pixles x 36bit(colour res) x 30fps = 279Mbytes/sec An 8 drive SATA raid with 250MB Maxtor MaXLine III’s will yield 38.2MBx8 = 305.6MB/s (at it’s slowest inner tracks) which is 1.8hrs of this uncompressed video. X-Raid does this with an external enclosure and via Fibre Channel (400MB/s) to a card in the PCI-X in the Mac. There seems to be other similar cheaper non-apple options as well. SATA-II seems to do this cheaper again when it comes. So IMHO storage is do-able. Larger faster HD arrays can be built also [size=+1.5]2.Camera[/size] This is the biggy, what camera will output uncompressed, unaltered 4:4:4 10 or 12bit 1920x1080 30fps data (with usable lenses for shooting, to preserve resolution, cheap is good) to a computer or straight to the hard drive array. Cameras that I have seen discussions of Summix Imperx Silicon Imaging Redlake Kinetta (but this seems to far away) Sony X300 (seems more expensive than the rest) Issues Connection to comp or HDD: HD-SDI...............too slow Dual HD-SDI...............Fast enough Firewire 800...............Too Slow Cameralink...............Fast enough Gigabit Ethernet...............too slow Drivers for the Mac??? Interface card for the Mac???? Aja's KONA 2 Dual HD-SDI Decklink HD Can you connect straight to HDD Array??? Format A Format than can retain the info and be read or converted to a readable form (without info loss) for FCP HD CCD’s Larger CCD better??? 3 better than 1???? Is there a camera that can do this, if there are multiple which one gives a better picture, f-stops, lenses etc. Suggestions for a setup is what I am looking for

June 21st, 2004, 10:28 PM	#2
David Newman CTO, CineForm Inc. Join Date: Jul 2003 Location: Cardiff-by-the-Sea, California Posts: 8,095	If you are considering any of the single chip Bayer solutions your data rate is a third of what you have calculated. If you intend to pre-process Bayer data there is little advantage to producing 4:4:4 data as you really only have an approximation closer to 4:2:0. Split the difference and use 4:2:2 like most of the industry then HD-SDI will work perfectly for 1920x1080 4:2:2 10bit @ 30fps. __________________ David Newman -- web: www.gopro.com blog: cineform.blogspot.com -- twitter: twitter.com/David_Newman

June 22nd, 2004, 04:40 AM	#3
Rob Lohman RED Code Chef Join Date: Oct 2001 Location: Holland Posts: 12,514	I'm not sure where you are getting at with this. What you want is simply not possible. Since you want it cheap. This is my humble opinion on the matter. The datarates are way too large for anything to handle that is CHEAP. But you say we have cheap RAID controllers these days. Yes we do. The only problem is that the normal PCI bus cannot handle these datarates. So everything needs to be PCI 66 which futhers adds to cost. Then I'm not sure how you would edit this (in what format). Keep in mind that camera's from Silicon Imaging cannot record to harddisks etc. As you probably know we are working on such a system now, but it will do 1280x720 10 bit to begin with. I've said it before and I will say it again. I'm not sure why everyone is so obsessed with HD and 10/12 bit recording. Would it be nice to have, definitely. But a lot of people do not seem to have a clue as to what it entails as far as: 1) costs (camera + interfaces + software + storage + computer + monitor etc. etc.) 2) portability 3) ease of use (ie there is no camera white balance, digital effects etc.) 4) BACKUP! (people already have problems with DV backups!) 5) editing problems (size, support etc.) 6) extra time and costs on your set (need monitor and probably other support systems in place as well) 7) lenses (research, buying etc.) 8) power on set Then again I'm seriously doubtfull on how many people will actually shell out the amount of cash (think at least $4000 for the full system) for a homebuilt system with basically NO SUPPORT or WARRANTY (if you build your own!). Yes we have a community, but that's a different thing. The system will probably be even more expensive if someone where to build and sell. This due to manufacturing, research and support costs. And this is only for 10 bit 1280x720 I'm talking about now! I'm not trying to scare anyone away, but I would like to add a few realistic (in my mind) pointers to all of this. Let's start fairly (and I don't say this with ease, but it ain't easy!) easy and see where this and the market takes us. Don't forget to write a good story first.... to shoot with this magic cam I will also see where this thread takes us because we already have 3-4 (!) threads running on this subject. It get's very hard to follow what is going on with all these threads, especially for outside companies trying to support this all. I understand the need for discussing certain problems in more detail, but in my mind we just are a pretty long way from discussing the things proposed at the beginning of this thread. __________________ Rob Lohman, visuar@iname.com DV Info Wrangler & RED Code Chef Join the DV Challenge \| Lady X Search DVinfo.net for quick answers \| Buy from the best: DVinfo.net sponsors

June 22nd, 2004, 05:50 AM	#4
Chris Hurd Obstreperous Rex Join Date: Jan 2001 Location: San Marcos, TX Posts: 27,368 Images: 513	I'm not sure that I understand the need for capturing uncompressed, unaltered, 4:4:4 twelve-bit 1080i video. How are the current HD formats (such as DVCPro HD) inadequate? How can you actually see any difference. Compression is not a bad word. __________________ CH Search DV Info Net \| 20 years of DVi \| ...Tuesday is Soylent Green Day!

June 22nd, 2004, 06:28 AM	#5
Steve Nordhauser Silicon Imaging, Inc. Join Date: May 2004 Location: Troy, NY USA Posts: 325	Chris: I agree, there is nothing inherently wrong with compression. It carries two evil threats that are not necessary but always lurking. The first is loss of real information that will never be recovered. That is why it is universally agreed that lossless compression is acceptable (barring the other evil), "virtually lossless" (hard to define other than with respect to the image noise floor) is probably fine, and somewhat lossy may be OK for the final release but not for the raw recording. The second 'bite your butt' is processing time. Obviously, you need to record fast enough not to drop a frame. Even with buffering, the average rate out to disk must be able to meet the average in from the camera. Fine. Compression helps that. But now, instead of data going by DMA (direct memory access) to a disk controller, it goes to system memory, the processor does the compression (this is why simple is better in real time) back out to memory and written to the HD in blocks. Some people are doing this, depending on the camera clock rate, bit depth, etc. Cineform and others seem to have pretty high clock rates they can handle. I'm sure you need a faster CPU but maybe only a single fast HD - important for the removable pack people. __________________ Silicon Imaging, Inc. We see the Light! http://www.siliconimaging.com

June 22nd, 2004, 06:44 AM	#6
Chris Hurd Obstreperous Rex Join Date: Jan 2001 Location: San Marcos, TX Posts: 27,368 Images: 513	Thanks, Steve -- it's great to have you here, by the way. __________________ CH Search DV Info Net \| 20 years of DVi \| ...Tuesday is Soylent Green Day!

June 22nd, 2004, 06:52 AM	#7
Rob Lohman RED Code Chef Join Date: Oct 2001 Location: Holland Posts: 12,514	Chris: As Steve excellently explained it is a tradeoff, isn't it always. I can clearly see DV compression on more complex scenes. So that's a bit too much. We are definitely working on getting some form of compression (lossless) in the system. The problem we have (as some points pointed out to by Steve): 1) we want as less amount of harddisks as possible. This requires compression 2) we have a limited amount of resources available to do this compression since we want to go smaller in the future (less CPU speed) and it is much data we need to handle 3) a lot of compression algorithms / codecs can't be afforded by us to license like some of the higher ends camera's can do (or develop themselves) 4) 99% of the algorithms out there are for RGB or YUV data at mostly 8 bit (that's probably 75% or so). We operate in Bayer. In regards to Bayer. This actually helps us (not in quality, but in data bandwidth needed). We get a 1.5:1 reduction already coming from the camera. Then if we store it efficiently we get a reduction of 1.8:1. This is without any quality loss (the signal from the camera is stored as it is coming in, only stored more efficient) and no compression. So with compression we do hope to bring this down even futher. Althought we can't make it lossy due to it being Bayer. The Bayer format needs to be kept intact to do the best Bayer to RGB/YUV conversion we can do. This process will be done as a form of "capture" stage on the computer. Since there is both time and more processing power available there. As Steve indicated we can't use DMA. So it is all a tradeoff... __________________ Rob Lohman, visuar@iname.com DV Info Wrangler & RED Code Chef Join the DV Challenge \| Lady X Search DVinfo.net for quick answers \| Buy from the best: DVinfo.net sponsors

June 23rd, 2004, 05:46 AM	#8
Laurence Maher Join Date: May 2004 Location: denton, texas, usa Posts: 416	My personal take on the matter is this: FCP HD does at best 4:2:2 1080p (virtually) uncompressed via SDI. So really, you don't need a camera or storage capable of doing any more than that if you're working with FCP HD. I am just about to get a G5 with FCP HD with the later intention of getting one of these upcoming silicon imagin cameras (1080p due out within 12 weeks) and using it with some form of SDI or camera link card. I'm looking at recording a feature at 4:2:2 1080p around 200 Mbps, which is far from uncompressed, but is much better than the CineAlta or Varicam. Personally, I think this is the sweet spot for cost and quality. Going lower will get you a flick that looks too scarily close to "video", and going higher will break your bank book. As long as I can fool most people and have decent latitude, I'm okay. The rest can probably be made up for via the way you shoot it (ND filters, etc.). I do agree that 1080 4:2:2 is close to a necessary evil, but then again, if we can get 720p at 4:4:4, that's pretty much gonna compete with it, if not be better.

June 23rd, 2004, 07:45 AM	#9
Rob Lohman RED Code Chef Join Date: Oct 2001 Location: Holland Posts: 12,514	Well, these CMOS chips basically do 4:2:2 already. Or that's how you can look at it. It is NOT 4:4:4 unless what everyone is writing. So those datarates are still a lot. One thing I was wondering. I thought HD was 720p or 1080i (notice the interlaced inidicator there!), according to standards. Anyone know for sure? __________________ Rob Lohman, visuar@iname.com DV Info Wrangler & RED Code Chef Join the DV Challenge \| Lady X Search DVinfo.net for quick answers \| Buy from the best: DVinfo.net sponsors

June 23rd, 2004, 04:03 PM	#12
Valeriu Campan Major Player Join Date: Mar 2004 Location: Melbourne, Australia Posts: 223	A Viper camera control panel with its HDs are looking like 10 G5 put together. Not to mention the 21" monitors ASO...

June 24th, 2004, 12:36 AM	#13
Laurence Maher Join Date: May 2004 Location: denton, texas, usa Posts: 416	If I have to do that to get my film shot, you bet I'll drag it (VERY CAREFULLY!) But others are talking about putting together a cheap PC that I might use for capturing, then transfer the files to the mac. We'll see. Any way you look at it, it's very true that real filmmaking (especially theater-quality HD) is extemely bulky. The idea here is not to get around the amount of work necessary for a great film. The idea is the cost difference. HD will end up being much cheaper with these new camera systems and computer storage than messing with the costs of film. (Well, if you do it right).