Juan,

Vegas Video can handle it perfectly. So can insync's blade 2. Although, I was very disapointed with my purchase of Blade 2 when it first came out. It was like installing an elephant onto my system. Of course, that was when it first came out and when it was the only software that could handle the advance 24p pulldown on the camera.

John

Juan, I just sent you a jpg to point something out that hopefully will be helpful. I haven't been able to read through the entire thread to see if this was covered, but it was only something that became apparent when I drastically altered the contrast of the RAW image. I hope it is useful.

Juan: could you tell me which device you are using to allow
connecting a straight firewire harddisk to the camera? I'm
thinking about experimenting with some things myself this year
and could really use some pointers on such a thing.

Obin: those guys seem pretty cool. I'm assuming they are
expensive?

I am not sure about this but those uncompressed capture cards for Apple such as the aja or the black magic decklink claim to capture to a 10bit video format. The only problem is that I think these are YUV 4:2:2 format files. I was just thinking that it must be possible to create a video format that at least used 10bit.

Also apparently these files work great in Final Cut Pro so that must mean that Final Cut Pro can handle higher than 8 bit video.

The windows xp drivers for the black magic decklink card that works with Adobe Premiere Pro however currently only works at 8 bit video so this may mean Premiere Pro might have a bit depth limitation. The company is trying to get 10 bit to work in Premiere however so maybe it is possible. If 10 works than so should 12 bit. I hope this helps somewhat.

Vegas Video can handle it perfectly.

Despite being a Vegas user and fan, at the moment this is untrue. Vegas works in an 8bit color space, but can do 4:4:4 uncompressed up to 2k resolutions. Naturally you don't get to preview this kind of media in realtime :D

Incidentally, Vegas can read/write to 10bit codecs, but internal processing of the codecs, and the effects you choose to apply to them, all happen in 8bits.

Now that Vegas 5 is out a lot of this looks to be changing shortly -- Decklink was doing SD over firewire from Vegas 5 at NAB, and it's only a matter of time before the bugs are shaken out and this feature will be added to Vegas (likely as part of an update to 5.0, ala 5.0b or 5.0c).

Similarly, Douglas Spotted Eagle mentioned discussing 14bit (yes, 14!!) developements and their implementation with Vegas at NAB.

- jim

I stand corrected. Mostly, though, I was referring to Juan's question of whether you could even open 4:4:4 12bit tifs in Vegas and whether it was resolution independent and could handle odd resolutions like the images he's pulling off the CCDs. But I guess "perfectly" is a bit of an overstatement. I do love my Vegas though!

That's really exciting about the Vegas 5 developments. I'll be sure to stay posted on that.

John

Ok, so i dropped my last raw frame into FCP. I rendered it with compression set to 'none'. The output looks extremely close to the original image, but there's something about it...the noise speckles don't look quite as strong now. I'm not sure if it's a difference in the pixel aspect ratio of the viewer vs photoshop, or if some decimation occurred. I can't really say any latitude was lost....

Anyone have an idea of how to test for this, other than looking at it? <g> Maybe if I make a 1 frame long video? The sizes should be approx the same..? Apple is so unclear as to how the data is dealt with.

Juan

Bah. Now that i look at it, the speckles look 'doubled'....like in the original image hey are a single pixel, but on the rendered 'uncompressed' video they occupy two side-by-side pixels and look averaged....looks like 4:2:2 to me.

Has anyone done film editing on FCP or another program? Maybe we can use the DV footage as editing footage, and create an EDL for the raw frames?

Even though I own a copy of combustion, I'm no film editor, so correct me if I'm wrong here, but according to the manual, combustion works with and renders up to thirty two bits per channel, and has the ability to work with Look Up Tables, which allow one to handle film footage--and, presumably, 12 bit video--without a display that can actually show the stuff properly.

That, coupled with the new "edit" operator in version 3 and sub-thousand dollar price point (more like eight hundred, depending on where you shop), makes combustion seem like a potential client for this project...right? Are there considerations I'm not, you know, considering?

Gotta read that manual some more...

Well, i'm actually not sure now that the problem is decimation...if i open the raw frame in photoshop i can see it fine, but when i drop it in FCP. it automatically does some correction such that the image is nearly square..

i know, i know about the NTSC pixel ratio but that's not it(afaik). I set the sequence to the exact frame size, 773x495 and ~square~ pixels, but for some reason it has black bars on the sides and it compresses the image horizontally.

Anyone have a similar problem with premiere, vegas, etc?

Juan, instead of working with the "uncompressed 10bit" sequence preset, I found out that it is possible to create new presets using any of the codecs supported by quicktime. If you go and look at the options available in the compression settings, you can see the the uncompressed 10bit video codecs are actually labeled as "uncompressed 10bit 4:2:2"... so that clears one point.
I tried setting a sequence to use the TIFF codec and frame size 773x495 square (with TIFF all the rendering occurs in the RGB space, check out the video precessing tab!), and then imported your frame..... and the result was perfect! no black bars and it was identincal to the one in photoshop.
hope i helped in some way......

We may have to treat the RGB frames the same way film editors do. Every codec I could find is 4:2:2. I did find a Kona card that claims to capture video as 4:4:4 but it was for HD. There is also the bluefish 4:4:4 cards but I heard a lot of complaints about them.

Combustion may be a good way to handle the files however since version 3 does have basic editing built into it. Basic editing might be all that is needed for a movie and everything thing else like titles and color correction are there as well.

The capture software you make could create a new directory for the frames every time a new recording starts. This would help organize the frames better.

Juan, your aspect ratio issue in FCP is easily fixed (i think) - open your clip in the Viewer (double-click on it while it's in the timeline) and go to Motion -> Distort -> Aspect Ratio - make sure this is set to zero. for some reason, when presented with footage that has little FCP-friendly metadata about aspect ratio or pixel dimension, it kind of guesses, and makes adjustments in there. hope that helps.

That did the trick! Thanks Benjamin! It was set to 33.33 and thus doing some weird resizing on it.

it is my best judgement that when i set compressor to "None" or "TIFF", the output video looks ~exactly~ like the input frame, thus full color RGB(4:4:4) 16bpc. This is good...

Juan, there is a QuickTime codec called None16 (digitalanarchy) which is 64 Bit when used with After Effects or 32 Bit when used with QuickTime. It is crossplatform and produces great images. I did some testing with the .psd file and it looks great. It also runs great (no dropped fremes) on a dual gig G4 with FCP HD.
The raw frames could be imported into QT Pro as image sequence and then exported with the none16 settings.

Edon

Rob, yes they are and they will not sell direct to end users...they work with big accounts only...too bad

Juan, are you planning on releasing the pinouts to the A/D converter and, maybe, a basic schematic of the whole circuit? You know, in case you die in a nuclear explosion (god forbid) or someone else can take a crack at capturing their own (far more likely)?

Yes, that is a good idea. Maybe somebody else could just assemble a simillar circuit to check if he/her has the same noise speckles problem in their DVX100.

I'm curious, since the CCD's seem to be slightly offset....can we use this fact to create a higher resolution image by taking into consideration that the 3 ccds record slightly different areas of the image? I think some cameras offset CCD's slightly to get high rez pictures, someone told me the Panasonic 953 does this for stills.

Jon&Milosz:
I might release all the details, depending on where this all ends. However this is just not something that someone can just put together overnight, so it wouldn't really help. However, if you really want to know the inner workings of the DVX and can read schematics, purchase the service manual for the DVX...that's the only thing I used.

Juan

Juan, you got me fired up...I am now working on converting a 16mm film camera to HD at 24fps, 30fps and 60fps....it's going to work...I am sourceing the parts right now...I will keep you posted.....

However, if you really want to know the inner workings of the DVX and can read schematics, purchase the service manual for the DVX...that's the only thing I used.

Really? I thought you mentioned that the only board missing a schematic was this one. Or you mean you just figured out how it worked from how the stuff around it worked?

There are some boards missing from the schematic, however i think i might've mispoken because what is missing is the board diagram for exactly the board that has all of this on it. So the schematic for that specific area is there, even though other parts are missing, but the diagram that shows you where everything actually is physically is missing. Also, even though the schematic for the A/D's is there, all it has is lines going out of the page into sections of the board that are NOT in the manual, so it was a pain indeed.

This might all be a misprint in my manual however.

Done with my final computer vision project...wuf.

Update:
My lab just happened to get a shiny new Agilent 20Mhz signal generator with external sync, so I am going to use it to follow-through the clock, and i'll be able to find out if the speckles are due to over-loading the clock signal.

In either case i expect to have 4:4:4 10-bit Uncompressed clips up tonight in TIFF frames. It the speckles are still there, I will just post raw frames and let you guys do what you want with them.

I'm still not terribly sure about the alignment so i could post independent R,G,B frames also, other wise it's my best guess.

Do you have to align the r,g,b? I know they are off but I would rather have the raw original data without any processing at all.

I can adjust the alignment myself as a post process. This would be true for the software you write for the prototype also.

Ok, in this case I can post separate directories with the R,G,B frames for the clip...does everyone have the capabilities to put this together or should I also post RGB frames?

Well, if photoshop is the only image editor that will view 16 bit TIFFs, then anyone who can look at it can manage the three different channels, too.

right, but i figured putting together a clip at 24fps is annoying in photoshop if you have to go frame by frame and assemble R,G,B channels....i can do it with R,G,B clips automatically in shake but i don't know what other program does it.

I hooked up the function generator as a clock, and captured images without using the clock on the camera at all. The images have the exact same speckles, so that rules the clock out. I am going to try a few other things and see if I can figure anything else....

Putting them together as single images in RGB channels is fine but I would prefer them not repositioned to align them. Wouldn't subpixel offsets alter the data? I would rather have the images as clean off the CCD as possible.

I'd like to understand the prototype you are building. Let me see if I have the specs right.

1. A small external device. Possibly velcroed or taped to the camera.
2. From the unit a ribbon cable will run into the camera. Some mod-ing to the camera body to make a gap to allow the cable in.
3. Inside the camera contact connectors will be attached to the three CCDs. No soldering. They will 'clip' on and stay firmly in place. There is enough space in the camera to allow these connectors.
4. The camera will operate as usual. Recording to tape with all other operations intact.
5. The external unit will have a Firewire 800 port connected to a PC.
6. The PC will have software to capture the stream of data coming from the CCD.
7. The PC will need relatively fast and large disk space to capture all the data.

Does this sound right? Did I miss something?

How will the external box be powered?

How long can a firewire 800 cable be run?

Close, with a couple of differences:

I'm pretty sure the device box will mount using the tripod mount on the camera, and the box itself will have another tripod mount so you can still mount the entire thing on a tripod. i'm planning on the box having the same matching lines/color/switch style as the bottom of the DVX.

You can also connect the device directly to ~any~ FW800 hard drive for recording.

I'm not 100% sure for FW800, but i have a long standard firewire cable that's at least 10 feet long. I don't see why FW800 can't be just as long, and possibly longer. These cables have twisted pairs such as they are pretty safe from interference and long cable lengths are possible, just as with Ethernet wires.

However, if you connect the device to a portable drive, the drive can be mounted near the camera or carried in a shoulder bag. The files will be recorded in some format ready for NLE, like TIFF RGB frames. All alignment parameters can be set on the box itself, so it records them as you wish, etc.

Oh and another thing...i'm not completely sure, but i think the flex cable i am using is so thin, that no modification is needed to get the cable out, it fits in the small space between the two magnesium shells on the bottom. In either case, the modification needed would be so small, it's negligible. We're talking less than 200 micrometers mm thin.

Juan

It could be very simple and elegant to design it to fit between the camera and tripod. It will need to be very sturdy to allow for all the potential weight a camera might carry.

I have a 25ft standard firewire cable I have used with my DVX100.

All the software will be in the unit?

What will power the unit?

Juan.. i snapped two quick shots of these at NAB..

are these the type of HD CMOS sensors you referred to earlier? i kept my eyes peeled for anyone at NAB with similar solutions to the project you're working on now.. but i saw nothing, just a lot of expensive cameras, jibs, and broadcast stuff hehe.

#1 (http://www.adamgeek.com/nab/DSC05360.JPG)
#2 (http://www.adamgeek.com/nab/DSC05361.JPG)

yep! Actually, i'm in the process of purchasing 3 ProCam 2560's...will take a while though.

cool let us know how it goes. i will definitely take one of your devices for my XL1S (waiting patiently everyday hehehe), as will a few people i know, probably.. and at least one wa svery interested in your comments about "building" and HDcam to shoot to HDD. (the main problem as i perceive it, to stream to HDD an HD signal, is that you are moving so much data you either need to compress it with a codec before streaming, or move it through gigabit ethernet)

So, after several hours of testing with the signal generator, i haven't solved the speckles but i think i'm a little closer.

First of all, using the singal generator as a clock, doesn't get rid of the speckles in the image, BUT the image looks quite cleaner...like the edges don't have bumps which seem to be mis-timed pixels.

This leads me to think that indeed the problem is just plain old noise in my test setup, and that by using the signal generator i got rid of the noise in the clock wires but of course, not on the signal wires.

I have posted a photoshop file for a frame i just took. it is uncorrected, uncompressed 4:4:4 10-bit, and it still has the speckles, but i wanted to illustrate how much cleaner some edges look...specially the section of the mobil-1 oil box in the picture that doesn't have speckles on it. If you can imagine the entire image being this speckle-free, that's one incredible SD image!

http://expert.cc.purdue.edu/~pertierr/cap6_RAW.psd

I am in the process of getting my cat to sit in front of the camera long enough to take a short 3 second clip to upload. I will upload it completely raw so you can do as you wish.

Awesome... Juan, looking at your raw footage always reminds me of the images sent back by the Mars probes from the 70s. Maybe once you figure out the noise problem you should have a "Mars Lander" button that puts it back in for crazy people like me. :)

Great image! At least in terms of quality. The dutched cinema is reminding me of the batman series.

The RGB channels seem to be lining up much better. Is that because of the clock?

And why is the image now 775x496? It seems to vary, especially in X.

The lining up is not because of the clock directly, but rather indirectly, because the edges are crisper so I can line them up better in photoshop.

Also, the varying size is just my shaky photoshop hand when selecting the image portion to crop. :)

CLIP:

Ok, so i just captured a ~3 second clip. Since you wanted separate R,G,B files, i'm trying to upload the raw R,G,B files out of the camera, but since these images contain dummy pixels(they look like film strips) each one is 114MB only for 3 seconds! That's 342MB for 3 RGB seconds with the dummy bits.

Anyway, my cable modem is down to a crawl, so i'll leave it running and post here when(if?) it's done.
The best solution though is to modify my program so it spits R,G,B frames out as an option, but i'll do it tomorrow....too late. :)

In any case, we're looking at ~150MB for 3 seconds without the dummy bits. The one pitfall here is that we are forced to use 16-bit images when we are only using 10-bits(or 12). Maybe it is time to move over to Cineon.

Juan

No way! Capture the images to raw 10bit and then have a program that can convert the raw images to TIFF, TGA, Cineon, JPEG, QuickTime, whatever people want. That way you use the absolute minimum disk space as you're capturing, and you give people flexibility in outputing...

Capturing straight to 10bit linear Cineons is a surefire way to make your wonderful progress go to waste...

- ben

I thought Cineon could do RGB uncompressed linear 10-bit/12-bit at any res? Why would my progress go to waste?

Update:
I've been trying to upload the files but no success...each one is to take about 2hrs over my home connection but it always gets interrupted, so i am going to try and upload them using my connection at work today...

Juan

ok, i'm no video specialist but i am supremely amazed.

This is a (approximately) 720P HD version of the last frame i posted, up-rezzed with PhotoZoom pro(S-spline) but other than that completely uncorrected(i.e. color still looks weird). I used the 'soft' photo setting which minimizes sharpening. I have a huge worklight lighting the scene, so the edge of the silver case is clipping the CCD's in the raw footage, but in the DV footage the entire silver case, white clothing and half of the oil box is clipped.

S-Spline understandably has a lot of trouble with the speckles, because it thinks they are details and they endup looking worse, but check out the areas that are speckle-free...wow! Since the original image is a strange aspect ratio, it resized to 1240x794. Specially details like the mobil-1 logo and the edges of the TV look pretty darn near perfect.

http://expert.cc.purdue.edu/~pertierr/cap6_HD.tif

Cineon technically can store uncompressed linear 10bit data. But I don't know of a single application that would be able to open such a file. If you go that route, be prepared to have people shrug and walk away...

- ben

Hmm interesting...all the apps i use, Apple Shake, FCP and Photoshop open cineon files. So, does Adobe AfterEffects/Premiere or Vegas not handle them?

I'm only asking because I did some tests and indeed it can store the exact same uncompressed information but with 10-bit or 12-bit packed data, reducing the file size by at least 25%, yet mantaining the exact same data as a 16-bit TIFF or RAW file.

Cheers,
Juan

After Effect yes, Vegas, no (unless added in v5), Premiere no, Premiere Pro maybe.

Thanks Stephen...i guess all of this can be made an option. Cineon will keep the same quality but will allow to get more video on the drive.

Unless there is another widely used standard that allows 10/12bpc depths...i'd like to keep it an option for those who have software that can handle it.

CLIPS:
The raw 'film strips' are being uploaded, the RED channel is already uploaded. The other two should be done in under 2 hours. To open the raw files, open the file in photoshop CS as a RAW file, using the following settings:
Width:1001
Height:59900
Depth:16-bit
Byte Order:Mac
Header Size:800
Click OK, it will tell you the selected image is smaller than the file, yadda yadda(always does this), click OK again.

Remember that the blue images are flipped horizontally, and as of this post only the RED is up:

http://expert.cc.purdue.edu/~pertierr/R_OUT16.raw
http://expert.cc.purdue.edu/~pertierr/G_OUT16.raw
http://expert.cc.purdue.edu/~pertierr/B_OUT16.raw

Juan

Let me get this straight -- Shake, FCP and PS opened 10bit linear Cineons? I would be shocked to hear that they all supported an entirely unused format -- 99.999% of all Cineon files are 10bit log, since that's how they come out of the film scanner, and that's how they go to the film recorder...

The reality is that Cineon in general is just not supported in many applications, and I think it doesn't make sense for an application such as this... I still think the best route is to record raw 10/12 bit files, and provide a conversion app. That way you don't need to do any processing on the data as you write it to disk...

- ben

That's a good suggestion, Ben...just record packed 36-bit data to maximize disk usage, and then provide an app that decodes it just like i've been doing.

My current final design does everything on the fly, but i hadn't considered that if everything is decoded into 16-bit RGB files for NLE instead of 12-bit files, there is a lot of valuable unused space on the drive....hmmm.

All R,G,B raw frame strips have been uploaded at the links above.

O happy day.

I have gotten rid of the speckles!

Now taking suggestions for test scenes/clips.

It was indeed noise in the lines as predicted a few messages ago, and it was solved with the use of some coupling capacitors.

Juan

Great work Juan! How about posting a few noise free stills? It would be nice to see some outside shots.