4:4:4 10bit single CMOS HD project

[Jason Rodriguez]

Hey Guys,

Adobe's just introduced a proposed open universal camera RAW format file called .DNG

you can read about it here: http://www.adobe.com/products/dng/main.html

This might be something to look into. I know that we've been working on .IHD, but if Adobe's also coming to market with a proposed universal RAW format, then that might be something we'd want to get in on for future format compatability.

[Jason Rodriguez]

Also here's a big plus to supporting Adobe's RAW format:

We get to use all those NICE Raw converters out there like Photoshop's, etc. In other words, we don't have to hack our own.

And if this file format catches on, there'll be MANY raw format converters with $$$'s behind them for R&D, while ours may be nice with open source stuff (actually .DNG is an open standard), I'm not sure how many coders out there are willing to put in the dirty work and try to get the stuff to work right. With this we can get some great algorithms, custom algorithms, etc. that will come with nice converters fueled by the money to be had with the digital photo market.

Also it supports packed bits (big endian), so we can save 10 or 12-bit files.

Tell me what you guys think, but so far I'm thinking this might be MUCH better than quicktime, .IHD, etc., because of the money behind it, and the potential for a lot of industry support.

I'm not downing anyone's hard work on this list, only that a universal RAW format that's an open standard (and based on TIFF), will greatly help us get these little projects off the ground.

[Obin Olson]

maybe

[Rob Lohman]

I don't see from a quick scan how this would help us much since
I couldn't see a way to implement a real-time compression system
in these formats (if they support some compression I doubt it will
be real-time and lossless).

Do these formats support more than one image per file as well?

The basic idea (at least for IHD) was to have an interim format
that suits what we are doing and work as fast as possible until
it can be transformed into something else.

As far as I know there is no NLE program yet that supports
RAW or the DNG format yet.

[Jason Rodriguez]

How hard will .IHD be to interpret if we want to write our own converters-I mean just curious, how easy will it be just in case a couple years from now (6-7, maybe even further) there's no updated software support?

also the .DNG file is a type of RAW file. So instead of .IHD, you'd be using .DNG's and use raw conversion software that understands how to interpolate those files.

I'm just thinking that there are a lot of nice commerical demosaicing algorithms out there, including the one that's bundled with Photoshop that either already or will understand and work with .DNG files. So it might not be a bad choice to have the ability to use these commercial converters, and futhermore, if this format really catches on, there'll be more converters, etc. that'll take advantage of this format.

The NLE's will support the TIFF's, etc. that come out of these .DNG converters.

[Jason Rodriguez]

Rob L. and S.,

What are the specs on .IHD going to be? Initally I simply thought it was going to be a RAW file with some header information for reading the file, making it fairly simple work to create your own de-mosaicer, but with the inclusion of multiple frames and compression, this sounds a lot more complicated.

Are there any planned specs for this file format, and if so, how hard will it be to "roll your own" converter?

Jason

[Rob Lohman]

The format has reached version 1.0 so to speak and the "spec"
(although not in full document form) is done. It is still very simple
and contains the following:

1. a simple bare bones format to store multiple frames in bayer format in for now two formats

2. these two formats are packed bayer and a compression algorithm I am working on called IDC

Now this IDC compression is very simple, especially at the
decoding end. We are planning to release the converter source
as it stands now very soon and it will include a full C header
file for the IHD format including some documentation.

The IDC engine will not be there yet since I'm still working on it,
but it should be out if all goes well not too long after the initial
release. So everyone can take a look at it then.

As I said DNG/RAW are certainly interesting but I could not see
in the quick scan I did anything about more than one picture or
different compression algorithms.

It might be a thing we want to implement (perhaps not in the
camera head but lateron), but more research has to been before
going done another/different path.

Jason: why would multiple frames be more difficult to process?
This is what you need to do in the end anyway. And whether
you read multiple files from the file system or multiple frames
from one (or multiple) IHD files is basically the same. The main
engine to do this kind of processing is already done in the
converter engine. Basically it works with plugins to do de-bayering
and other image manipulation including output to the final format.

Just for everyone wondering how large these files can get we
have set the mark at 1 GB at the moment. This to facilitate
different (file) systems in the future.

[Jason Rodriguez]

Hey Rob,

I'm not so sure that 1GB file-size limit is a good thing.

AVID Xpress Pro on Windows has to do this with their OMF files and it's an extreme PITA!

Will there be a bit(s) in the header that tells us how many files are in the 1GB "big" file, or will we simply keep skipping to the next frame till there are no more?

I like the idea of single frames from the file system a lot more if that's still possible to keep.

Plus, if I ever want to match-back to a specific frame, I'm gonna have to search through some IHD file for an obscure frame, and I can't search through the file since it has to be converted first (it's not a quicktime), etc.-in other words there is not one-file-to-one-frame that easily facilitates going back to the original RAW frame should I decide to do window burns in an offline edit. Instead I'll have x amount of files with a single window burn from file XXXX.ihd, and have to count back the frames? Gosh, that sounds like something I really don't want to spend my time hacking away at.

Please again, if you can, don't embed more than one image in single IHD file if you can help it, that would help us out a lot more on the editing end and for archival and offline-online matchback.

[Rob Lohman]

As it stands now that is not the way Rob S. and myself have
designed it. The IHD system has intelligence to know which
files belong to which. Keep in mind that the IHD system was
setup as a digital negative or original negative so to speak.

It is not meant to be used for editing or any other processing
except to convert it into another format. It is not in full RGB, it
is in bayer format, which an NLE cannot handle.

The original idea was to have a convert application that converts
these original negatives to something you can work with. My
idea is to be able to output multiple formats in one go. So it
can output 16-bit TIFF sequences, and a DV file and at the same
time a highly compressed quicktime (low-resolution for example)
file to send of to other people who need to see what you
recorded with the camera.

So then you have (for example):

1. 3 files with the exact same naming convention so you know which scene it is for example

2. a low-end file to send over the web for example to other people who need to see the dailies

3. a DV "offline" file to start editing and create an EDL

4. a high-end 16-bit TIFF sequence for use in effects applications or conforming to the EDL

That's an "idea" for example. The idea is that a IHD file or sequence
(depending on length) will be created for each scene. So they have
an increasing number and we have been thinking about allowing
for custom scene numbers to be set or extra information to be
embedded. This way sets of files can be more easily identified,
BUT this will mean more work for each shoot with a camera
(you will need some way to input this).

The last thing above is not in the spec but there is enough time
to include the possability at least.

Ofcourse the way to the future is completely open. But for various
reasons (file system performance being one, which is VERY
important in our real-time camera system!) a single file per frame
was not on option in our minds. Again, this is not the final delivery
format as we envisioned it!

The 1 GB splitting is a non-issue since the convert application will
natively handle this without a problem as long as all the files are
present (which is true in a single image sequence as well!!). You
should never have to deal with this in the final format if your
system supports large files.

If the DNG/RAW format is something that is good for this system
it could first be implemented in the convert application to see how
well it works. Thats relative easy to do and performance is less
of on issue then as well. Then if it gets adopted and can perform
in a real-time environment it might be time to put it in the camera
directly. For now I personally don't see many benefits at this point
in time. But it is very interesting development!

[Wayne Morellini]

<<<-- Originally posted by Jason Rodriguez : I don't think that's "sheer", I think it's more like "tearing", like when you're trying to run a video game and it's not keeping up with the refresh on the screen.

If that's the case, then it might have to do with not using DirectX or the video hardware and trying to do too much in software. -->>>

I don't know how things are programmed here, but this sort of problem is also in 3d games. In the 3d control panel tab of my ATI card, their is a wait for vertical sync slider that canbe forced on or be set to "application preferece". So there must be a directx methord of using it. Using this with double buffering, mentioned elsewhere here, helps in program work flow to.

Hope this helps.

[Jason Rodriguez]

But for various
reasons (file system performance being one, which is VERY
important in our real-time camera system!) a single file per frame
was not on option in our minds

Okay, so the IHD file is created to take advantage of the speed of the file system, in other words, to get as many sequential writes as possible rather than random writes with single files? If so, that's a pretty smart idea, and it's sort of like mimicing a DDR in a way, although they get to stripe their disk file system in a manner that is the most eficient for their file format rather than the cludge of NTFS we have to deal with (not that NTFS is bad, but I don't like the fact that it's proprietary and there aren't any open source implementations or even abilities to mount and write to it-you can mount, but you can't write).

[Rob Lohman]

That is correct Jason. It also offers a form of prioritized RAID
when you have more than one drive to record to. The fastest
drive will get the most frames to make sure throughput is as
optimal as possible. Ofcourse getting two (or more) drives
of the same speed will be better and in this case frame 1 will
be written to drive 1 and frame 2 to drive 2 etc. to make sure
throughput is optimal for higher resolutions and framerates.

[Jason Rodriguez]

Hi Rob,

If you're planning support for Quicktime, could you also add support for higher-resolution Quicktime Codecs for the online format, like Microcosm, None-16 (both are 16-bit codecs), or the 10/12-bit RGB codec from Blackmagic?

[Rob Lohman]

I'm assuming that if you implement a QuickTime writer it will be
able to use all the codecs you have installed for it. But then again,
I might be wrong on that.

[Rob Scott]

Jason Rodriguez wrote:
kay, so the IHD file is created to take advantage of the speed of the file system, in other words, to get as many sequential writes as possible rather than random writes with single files?

That's correct, Jason. We're starting from the assumption that the drive(s) will be defragmented frequently, so we attempt to get the best performance possible by doing as few file open's as possible and doing lots of sequential writes. During capture, there are no seek's.