BTW, Obin and Rob S.,

What's the preview delay like on your apps? In other words, in quick motion, can you keep up with the subject, or is there a noticeable "lag" in the picture that's being previewed? If converting the image to color using a simple bayer de-mosaicer is slowing thing down, you may want to simply provide a way to get the black-and-white raw image to the preview screen as fast as possible. Image lag is a real killer, or at least can be when trying to be precise with your camera moves.

<<<-- Originally posted by Jason Rodriguez : BTW, Obin and Image lag is a real killer, or at least can be when trying to be precise with your camera moves. -->>>

This sounds like a good point. There will always be a little lag due to processing needs, but it would be great to know how many frames something can lag before it becomes noticeable or difficult to work with. I'm sure nobody will notice 1 frame, but 5?

Eliot

Jason:
I would think that a 1/4 scale preview by using quadlets (R,(G1+G2)/2,B) for color would be very fast and color accurate for the raw video - analog white balance but no gamma, saturation adjustment.

Hey Steve,

If it was possible though, I would much rather have gamma than color, because I'm not necessarily trying to use the viewfinder for color, but to adjust the exposure. With linear images being so dark, it would make it very hard to get proper exposure when you can't see what you're doing.

BTW, five frames behind would be an awful lot, almost to the point of unuseability for any fast work. One frame isn't bad, but five is not really acceptable.

That's why I'm thinking, if this becomes the case with color previews, etc. the simple thing is to just to the decimated raw image in black and white and a gamma adjustment on it or custom viewLUT (actually a custom viewLUT, even if it's a text file that the program is accessing and not a nice gui, is the preferred method of viewing. That way no matter what the exposure for the linear file, we could see what's happening. The best thing would be color preview and a custom viewLUT).

Jason why not adjust the gamma on your screen to counter the camra raw preview? I did it here and it worked greAT...just boost the screen gamma way up and it looks like your looking at a gamma fixed image in post...

Hey Obin,

That does work to a point, but it would be nice again to have a custom viewLUT, to sort of preview how you want things to look. A custom viewLUT could simply be a custom RGB curve (that adjusts all three channels the same, not necessarily differently for each channel) that's accessed from a tab-deliminated text file for values from and to (like an initial value of 10 gets mapped to 20, etc.).

Obin,

How did you latest version of CineLink turn out?

ok...we are having issues with re-draw because of the non-use of hardware for screen draw..the 1/4 screen color preview is great but the windows redraw is really bad..working on that now

Are you getting any delays (you know, when you pan, etc. the screen preview is behind the camera motion) in your on-screen preview?

BTW Obin, what format is CineLink outputting? Is it a RAW format, or something else with a custom header?

not much delay at all...I think that issue is fine
RAW data output

Sorry, don't mean to beat a dead horse, but would you say 1-2 frames behind in delay, or 5-7+?

best guess is 1-3 frames from what i have seen( I have been loaded with work and not much time for testing) I will do some things in the morning and see how it looks..and try the new bayer filter..we may work with DirectX for a faster screen refresh in CineLink..not sure yet..

Yah Obin, do some tests . . .

if it's 1-3 frames, that's quite good, especially if it's towards the 1 frame mark :)

BTW, what's the new bayer filter?

not even sure yet!

I will see today :)

Rob or Rob..why are we getting what I call image shear with camera movement like a pan on the preview with color bayer? its like the image is sliced up when you pan the camera..what causes this and how can it be fixed?

TIA for your help

1/4 pixel quad preview works well! but this image "shear" is bad...btw it is much better at 1/15 shutter speed...is this a clue?

I haven't got a camera so I can't see what it would be, but it
sounds like a simple rolling shutter issue? Although at 1/15 it
should be more worse, not less.

no no not that at all..it's an issue with redraw I think..

it is the screen not refreshing completely before the
picture is totally displayed and the next frame being superimposed on the
previous. I will try to force a refresh after dumping things to the screen
this might help somewhat.

...from Luc

You could try double-buffering.

I don't think that's "sheer", I think it's more like "tearing", like when you're trying to run a video game and it's not keeping up with the refresh on the screen.

If that's the case, then it might have to do with not using DirectX or the video hardware and trying to do too much in software.

<<<-- Originally posted by Rob Scott : You could try double-buffering. -->>>

Isn't double-buffering a hardware issue though? I thought Obin said he's doing this all in software, hence no double-buffer (unless you program one).

Possibly, but in software you still have to indicate when you're done with buffer A and are starting on buffer B. IOW, the system won't magically know when you're finished with one frame and are starting on the next one.

Hey Guys,

Adobe's just introduced a proposed open universal camera RAW format file called .DNG

you can read about it here: http://www.adobe.com/products/dng/main.html

This might be something to look into. I know that we've been working on .IHD, but if Adobe's also coming to market with a proposed universal RAW format, then that might be something we'd want to get in on for future format compatability.

Also here's a big plus to supporting Adobe's RAW format:

We get to use all those NICE Raw converters out there like Photoshop's, etc. In other words, we don't have to hack our own.

And if this file format catches on, there'll be MANY raw format converters with $$$'s behind them for R&D, while ours may be nice with open source stuff (actually .DNG is an open standard), I'm not sure how many coders out there are willing to put in the dirty work and try to get the stuff to work right. With this we can get some great algorithms, custom algorithms, etc. that will come with nice converters fueled by the money to be had with the digital photo market.

Also it supports packed bits (big endian), so we can save 10 or 12-bit files.

Tell me what you guys think, but so far I'm thinking this might be MUCH better than quicktime, .IHD, etc., because of the money behind it, and the potential for a lot of industry support.

I'm not downing anyone's hard work on this list, only that a universal RAW format that's an open standard (and based on TIFF), will greatly help us get these little projects off the ground.

maybe

I don't see from a quick scan how this would help us much since
I couldn't see a way to implement a real-time compression system
in these formats (if they support some compression I doubt it will
be real-time and lossless).

Do these formats support more than one image per file as well?

The basic idea (at least for IHD) was to have an interim format
that suits what we are doing and work as fast as possible until
it can be transformed into something else.

As far as I know there is no NLE program yet that supports
RAW or the DNG format yet.

How hard will .IHD be to interpret if we want to write our own converters-I mean just curious, how easy will it be just in case a couple years from now (6-7, maybe even further) there's no updated software support?

also the .DNG file is a type of RAW file. So instead of .IHD, you'd be using .DNG's and use raw conversion software that understands how to interpolate those files.

I'm just thinking that there are a lot of nice commerical demosaicing algorithms out there, including the one that's bundled with Photoshop that either already or will understand and work with .DNG files. So it might not be a bad choice to have the ability to use these commercial converters, and futhermore, if this format really catches on, there'll be more converters, etc. that'll take advantage of this format.

The NLE's will support the TIFF's, etc. that come out of these .DNG converters.

Rob L. and S.,

What are the specs on .IHD going to be? Initally I simply thought it was going to be a RAW file with some header information for reading the file, making it fairly simple work to create your own de-mosaicer, but with the inclusion of multiple frames and compression, this sounds a lot more complicated.

Are there any planned specs for this file format, and if so, how hard will it be to "roll your own" converter?

Jason

The format has reached version 1.0 so to speak and the "spec"
(although not in full document form) is done. It is still very simple
and contains the following:

1. a simple bare bones format to store multiple frames in bayer format in for now two formats

2. these two formats are packed bayer and a compression algorithm I am working on called IDC

Now this IDC compression is very simple, especially at the
decoding end. We are planning to release the converter source
as it stands now very soon and it will include a full C header
file for the IHD format including some documentation.

The IDC engine will not be there yet since I'm still working on it,
but it should be out if all goes well not too long after the initial
release. So everyone can take a look at it then.

As I said DNG/RAW are certainly interesting but I could not see
in the quick scan I did anything about more than one picture or
different compression algorithms.

It might be a thing we want to implement (perhaps not in the
camera head but lateron), but more research has to been before
going done another/different path.

Jason: why would multiple frames be more difficult to process?
This is what you need to do in the end anyway. And whether
you read multiple files from the file system or multiple frames
from one (or multiple) IHD files is basically the same. The main
engine to do this kind of processing is already done in the
converter engine. Basically it works with plugins to do de-bayering
and other image manipulation including output to the final format.

Just for everyone wondering how large these files can get we
have set the mark at 1 GB at the moment. This to facilitate
different (file) systems in the future.

Hey Rob,

I'm not so sure that 1GB file-size limit is a good thing.

AVID Xpress Pro on Windows has to do this with their OMF files and it's an extreme PITA!

Will there be a bit(s) in the header that tells us how many files are in the 1GB "big" file, or will we simply keep skipping to the next frame till there are no more?

I like the idea of single frames from the file system a lot more if that's still possible to keep.

Plus, if I ever want to match-back to a specific frame, I'm gonna have to search through some IHD file for an obscure frame, and I can't search through the file since it has to be converted first (it's not a quicktime), etc.-in other words there is not one-file-to-one-frame that easily facilitates going back to the original RAW frame should I decide to do window burns in an offline edit. Instead I'll have x amount of files with a single window burn from file XXXX.ihd, and have to count back the frames? Gosh, that sounds like something I really don't want to spend my time hacking away at.

Please again, if you can, don't embed more than one image in single IHD file if you can help it, that would help us out a lot more on the editing end and for archival and offline-online matchback.

As it stands now that is not the way Rob S. and myself have
designed it. The IHD system has intelligence to know which
files belong to which. Keep in mind that the IHD system was
setup as a digital negative or original negative so to speak.

It is not meant to be used for editing or any other processing
except to convert it into another format. It is not in full RGB, it
is in bayer format, which an NLE cannot handle.

The original idea was to have a convert application that converts
these original negatives to something you can work with. My
idea is to be able to output multiple formats in one go. So it
can output 16-bit TIFF sequences, and a DV file and at the same
time a highly compressed quicktime (low-resolution for example)
file to send of to other people who need to see what you
recorded with the camera.

So then you have (for example):

1. 3 files with the exact same naming convention so you know which scene it is for example

2. a low-end file to send over the web for example to other people who need to see the dailies

3. a DV "offline" file to start editing and create an EDL

4. a high-end 16-bit TIFF sequence for use in effects applications or conforming to the EDL

That's an "idea" for example. The idea is that a IHD file or sequence
(depending on length) will be created for each scene. So they have
an increasing number and we have been thinking about allowing
for custom scene numbers to be set or extra information to be
embedded. This way sets of files can be more easily identified,
BUT this will mean more work for each shoot with a camera
(you will need some way to input this).

The last thing above is not in the spec but there is enough time
to include the possability at least.

Ofcourse the way to the future is completely open. But for various
reasons (file system performance being one, which is VERY
important in our real-time camera system!) a single file per frame
was not on option in our minds. Again, this is not the final delivery
format as we envisioned it!

The 1 GB splitting is a non-issue since the convert application will
natively handle this without a problem as long as all the files are
present (which is true in a single image sequence as well!!). You
should never have to deal with this in the final format if your
system supports large files.

If the DNG/RAW format is something that is good for this system
it could first be implemented in the convert application to see how
well it works. Thats relative easy to do and performance is less
of on issue then as well. Then if it gets adopted and can perform
in a real-time environment it might be time to put it in the camera
directly. For now I personally don't see many benefits at this point
in time. But it is very interesting development!

<<<-- Originally posted by Jason Rodriguez : I don't think that's "sheer", I think it's more like "tearing", like when you're trying to run a video game and it's not keeping up with the refresh on the screen.

If that's the case, then it might have to do with not using DirectX or the video hardware and trying to do too much in software. -->>>

I don't know how things are programmed here, but this sort of problem is also in 3d games. In the 3d control panel tab of my ATI card, their is a wait for vertical sync slider that canbe forced on or be set to "application preferece". So there must be a directx methord of using it. Using this with double buffering, mentioned elsewhere here, helps in program work flow to.

Hope this helps.

Okay, so the IHD file is created to take advantage of the speed of the file system, in other words, to get as many sequential writes as possible rather than random writes with single files? If so, that's a pretty smart idea, and it's sort of like mimicing a DDR in a way, although they get to stripe their disk file system in a manner that is the most eficient for their file format rather than the cludge of NTFS we have to deal with (not that NTFS is bad, but I don't like the fact that it's proprietary and there aren't any open source implementations or even abilities to mount and write to it-you can mount, but you can't write).

That is correct Jason. It also offers a form of prioritized RAID
when you have more than one drive to record to. The fastest
drive will get the most frames to make sure throughput is as
optimal as possible. Ofcourse getting two (or more) drives
of the same speed will be better and in this case frame 1 will
be written to drive 1 and frame 2 to drive 2 etc. to make sure
throughput is optimal for higher resolutions and framerates.

Hi Rob,

If you're planning support for Quicktime, could you also add support for higher-resolution Quicktime Codecs for the online format, like Microcosm, None-16 (both are 16-bit codecs), or the 10/12-bit RGB codec from Blackmagic?

I'm assuming that if you implement a QuickTime writer it will be
able to use all the codecs you have installed for it. But then again,
I might be wrong on that.

That's correct, Jason. We're starting from the assumption that the drive(s) will be defragmented frequently, so we attempt to get the best performance possible by doing as few file open's as possible and doing lots of sequential writes. During capture, there are no seek's.

I believe that's correct, Rob, but I haven't tried it yet.

That looks interesting, but the site mentions only photography. Though I haven't read the documentation yet, I suspect that this format is optimized for still photography -- where a high frame rate is generally not a big issue -- and not for filmmaking.

Besides, if someone really wanted to use it, it could end up being very simple to convert from IHD to DNG.

Just curious then, why the 1GB limit when FAT32 has a 2GB limit?

Also do you think there will be a seek "hiccup" when the new file needs to be written, or is that what the RAM buffer is there to prevent?

Additionally, I think we'll need some sort of dropped frame indicator.

This all seems very exciting, but I'm wondering how far in the distance we're talking here, especially for Quicktime conversion.

It seemed like a convenient "chunk" to use, and prevents problems if using unsigned longs for seeking.
Exactly, the RAM buffer should smooth that out.
Yes, I have thought about that, but I'm not yet sure how I'm going to handle it.
The Convert software is very close to being released under the GPL; at that point, anyone who wants to can add QuickTime support. I have downloaded the SDK myself, I just haven't yet had time to dig into it.

I've worked with the QT SDK a couple of years back and it was
pretty painfull to figure out how everything needed to be done.
However, this was for reading a movie, not writing, which in
theory should be easier. After I'm done with my compression
stuff I hope to spend some more time with the convert application
and output formats and such.