4:4:4 10bit single CMOS HD project

[Matthew Miller]

Obin,
I second what Eric said. I'm anxious to see some footage too. WM9HD is beautiful stuff man. My projector is 1280x720 and I run DVI straight to it from a PC. I don't have the sheervideo codec, so I haven't been able to watch that quicktime clip. As far as I can tell, sheervideo is still Mac only. But I'd love to see what kind of images that sensor of yours gets blown up 84 inches wide.
Please! Pretty Please! Just even 8 seconds of something. Hell! I'll take 6 seconds.
I'll even take hundreds of individual 1280x720 images and compile them into a WM9 file for you. ( Matt begins whimpering like a child who wants its mother... it's 1280x720 mother)

[Rob Lohman]

Freya: it's quite easy

1. camera: SiliconImaging 1300 (Micron CCD chip)

2. interface: CameraLink => requires special CameraLink board in PC

3. software: streampix (comes with the camera). VERY lowlevel!

Rob S. and myself are figuring out how to our own camera
"firmware" or software. We both are quite busy at the moment though.

[Freya Paget]

I've been looking at these cameras!

The main streampix interface doesn't look so bad to me! I don't much fancy scripting tho! ;)

Presumably it is easy to grab frames but harder to grab actual video? Or maybe it all has to be done thru scripts and the windows interface is just a media player? What is easy and or difficult about the software?

So you are re-writing the actual camera firmware! Is that because it cannot continuously stream video at the moment, only a given duration at a time?

love

Freya

[Obin Olson]

Ok Matt! hear the call..I will try and edit some shots for you :)

guys streampix is not bad but it does not support the camera all you can do is capture, Xcap controls the camera and that thing is a bear even Rob hates it! and he writes software!

just wait around a bit and I think we will have a good capture program on the market before you know it!

[Rob Lohman]

Well, you've got to seperate things. SiliconImaging has built a
camera head that includes some chip and thus firmware. We are
not going to change that (at least as it looks now).

We are developing other software that runs on a PC that is more
geared towards how movie makers use such camera's. In essence
it allows us/you/whoever to turn a "PC" into a "camera".

The idea is to make it small enough so it is portable and allows
you to attach 1 or 2 harddisks (depending on shooting options)
and it will record straight to harddisk with things like viewfinder/
monitor out.

We are still a while away from that, though. I'm also gonna go on
vacation, shortly. I know Rob S. is currently very busy with other
things as well.

[Obin Olson]

Matt Promise you will give me a full on report of how this stuff looks on a 80inch screen? I have been dying to see this footage on a REAL HD monitor!

[Freya Paget]

<<<-- Originally posted by Rob Lohman : Well, you've got to seperate things. SiliconImaging has built a
camera head that includes some chip and thus firmware. We are
not going to change that (at least as it looks now).
-->>>

Oh! That's what I thought you meant when you said firmware earlier! Phew! Glad to hear you are not having to mess with that!

So it's easy to capture individual frames at the moment but not motion?

Is the software proving very hard to write?

Thanks for all the help and info! :)

love

Freya

[Rob Lohman]

Software and system development for such things is always
hard. Even the big manufacturers find that hard.

You can easily capture motion as well. It is just written out as
single frames which must convert to a movie. Remember that
a movie is nothing more than a sequence of individual frames.

Even a photo camera can make movies, albeit usually at a lower
framerate. ANY camera (sometimes with changes) could be used
for making movies in the technical sense of the word.

[Rob Scott]

Freya Paget wrote:
So it's easy to capture individual frames at the moment but not motion?

Capturing the frames is not terribly difficult. Given their size, however, the challenge is writing them out to disk in a timely fashion.

Is the software proving very hard to write?

I wouldn't say especially hard, but then again that's a relative thing. The biggest challenge for me so far is squeezing in the time to work on it! It's also time-consuming because I'm working with stuff I've never done before -- DirectDraw for example.

[Steve Nordhauser]

Freya:
Here is the rundown as I see it. We (Silicon Imaging) and others make cameras that are basically the sensor with some intelligence to control it (where the camera firmware resides) - small, low cost (relatively) and some basic interface. For speed, we have been a proponent of camera link, an industrial high speed interface because we can also supply low cost frame grabbers (capture cards). For the record, we also support USB 2.0 (not applicable to the speeds required here) and gigabit ethernet.

A software developer's kit (SDK) can get video (single or continuous) into PC memory if you are doing your own application. There are some off-the-shelf packages that can display and record. The two bandied about here are XCAP and Streampix. XCAP is more of a researchers' tool. Although the interface is not great for cinematography, it is actually easy to use and very powerful for other purposes like machine vision. Streampix is a recording package - very good at moving continous video to disk. Post processing is pretty weak - they are improving their Bayer algorithm on Obin's request and can save in a bunch of formats. Both of these packages can either save single frames or large binary files of continuous video. This is done to speed the recording time - no preprocessing at all.

The good thing about these packages and hardware is that they are 8/10/12 bit aware. The gang here is developing a replacement for recording and file formatting so that standard NLE applications can then be used. They are also working on the hardware packaging to be as usable as possible - that involves cameras, computers, power, etc. I'm doing my best to help support these efforts since there is a lot of work being done to help a community of people by what seems about 6 people or so. I think the results have the possibility to bring incredible changes to a lot of indies.

I think that the challenges here are not in doing it, but in doing things at least as good as a professional result requires. This is in the understanding and implementation of: integration of the camera head into a suitable camera body (lens and DOF issues abound), a good viewfinder, disk recording without dropping frames using a minimum of disks and $$, a simple but complete user interface, a small but powerful enough computer, Bayer and color processing that provides the best quality image, possible compression to reduce cost and size but not compromise image quality, and the smooth movement of the capture video to NLEs.

There are people with varying needs here, from 1280x720@8bits/24fps to 1920x1080@12 bits/30fps - very different solutions are required.

I'm sure that I left things out, but I think that is the charter of what is going on in Alternative Image Methods in about 3 or 4 threads.

[Freya Paget]

Wow thanks Steve and everyone else who tried to bring me up to speed.

It doesn't sound as bad as I expected at all! It seems like it's mostly just a case of interfacing with the windows multimedia API's to shove the data into a codec as it comes in!

It seems like one solution might be to adapt the code of virtual dub which would not only allow you to insert frames into codecs but also allow primitive editing!

Unless you are actually writing genuine windows drivers for the thing! Now that would be really nice! :)

It all sounds quite optimistic!

love

Freya

[Rob Scott]

Freya Paget wrote:
It seems like one solution might be to adapt the code of virtual dub which would not only allow you to insert frames into codecs but also allow primitive editing!

I love VirtualDub and looked into adapting it. The main problem is that it's hard-wired to 8-bit video only. We need to support 10/12/16-bit depths.

Windows drivers...

Yikes! :-) That's definitely outside the scope of the current project, and I don't think it is necessary for our purposes either.

Down the road, I think it would be cool to see if we can use the Dirac open-source codec to provide low-cost cross-platform visuall-lossless compression. It would also be cool to investigate if a high-quality Bayer filter and the Dirac codec could be implemented in a FPGA solution and embedded into the camera head itself. In theory, this could allow high-resolution, high-quality, high-frame-rate capture. Hard drive bandwidth space would be less of an issue; we might be able to get by with one drive. Plus, we could capture to a standard format in one step; no need for offline processing. Okay, I'm dreaming now ...

[Freya Paget]

<<<-- Originally posted by Rob Scott : I love VirtualDub and looked into adapting it. The main problem is that it's hard-wired to 8-bit video only. We need to support 10/12/16-bit depths.
Yikes! :-) That's definitely outside the scope of the current project, and I don't think it is necessary for our purposes either.
... -->>>

Would virtual dub have been hard to adapt in 8 bit tho? Even 8 bit hi-def would be nice!

Sorry about scaring you with the though of windows drivers. I didn't mean any harm. I once looked at the windows DDK but then I decided I had better things to do with my life such as eating a big pot of ice cream under the shade of a tree! :)

love

Freya

[Rob Scott]

Freya Paget wrote:
Would virtual dub have been hard to adapt in 8 bit tho? Even 8 bit hi-def would be nice!

That's an interesting idea, but I'm not sure how much it would buy us. While I'd love to have VirtualDub's filtering and preview engine(s), I'd also like to support Cineon, OpenEXR and QuickTime output, things that VirtualDub doesn't do.

I didn't mean any harm.

I'm better now :-)

[Matthew Miller]

<<<-- Originally posted by Obin Olson : Matt Promise you will give me a full on report of how this stuff looks on a 80inch screen? I have been dying to see this footage on a REAL HD monitor! -->>>

I Promise.
I'll take pictures of the thing if you want. (not that it would make much sense to do so.)