Eek, yeah, that's gonna be messy. . . first-off you can't really do effective trilinear on a scattered data-set . . . trilinear likes evenly spaced meshes. What would be more effective and accurate would be something like tetrahedral with only 4 points and do a triangulation of the mesh (like Delaunay). Secondly, the placement of the colors on the color-checker in device-independent color-spaces like CIELab are not beneficial for scattered data-set interpolation (at least not using literal interpolation, i.e, you need to move to some regression method), and typically what happens is that after triangulation you have tetrahedra that do not allow one to weight the points in the color-space appropriately for the weight of the point that is the training point . . . i.e., you can end up with saturated colors that are also interpolating through the mid-tones, so your mid-tone range gets all skewed as you move around the convex hull of the training points which are the colors from the ColorChecker chart.

In an ideal scattered data-set training sample, the colors are correctly spaced so that a nice evenly spaced lattice/solid can be used to create the interpolated data, but the ColorChecker series typically "clumps" samples around the convex hull of the color-space, and 4-points is not enough to weight the points in the interior correctly. So what typically happens is that colors that are near the exterior of the color-space and closely aligned with the color-checker points look fine, but then as you move into the interior of the solid, the points are all skewed incorrectly as there simply aren't enough points to create a nice interpolation lattice.

I think a better chart would be the IT-8 or something of that nature that creates a more "even" interpolation lattice with evenly spaced samples throughout the volume of the color-space, not clumps along the convex hull like the ColorChecker series.

So if I understand correctly that image then only has had gamma correction applied, there has been no saturation added to the image? For instance, you mentioned:

- Add rec709 gamma correction
- Convert to YUV using rec709 YUV conversion

So these two steps are not applying some form of color saturation multiplier on the image (they shouldn't, but was just wondering)?

I'm just wondering if that's the level of saturation you're getting straight from the camera head, or if there is a multiplier somewhere in your color-conversion steps to give the more saturated image I'm seeing as the end product. It sounds like from your description there isn't any saturation stages.

Thanks,

Jason

Indeed, these two steps are NOT adding color saturation.

There have not been any color saturation added anywhere, nor has any whitebalancing taken place, the colors are still in camera RGB.

The picture was taken in natural light from an overcast sky during noon in the netherlands.

BTW, I'd just like to clarify the "tone" of that statement . . . it sounds a little harsh and didactic, I wasn't meaning for it to sound like that . . . I'm sure trilinear can be done (and you obviously said it was working to some extent), but according to a lot of papers and sources I've read, scattered data-sets that are not evenly spaced tended to get better results from tetrahedral vs. trilinear interpolation.

Thanks,

Jason

Wow, that's pretty impressive then . . . it hopefully won't take you too much work to align those results to a proper color-space . . . a lot of your color vectors are already in the right spot or very close to them, so you shouldn't need any crazy matricies like those needed to uncouple sensors with a lot of color-channel cross-talk.

Thanks,

Jason

Hello Jason,

Well the color LUTs wasn't working yet, I thought I would be able to implement them quite easilly, just when I got to the point that is also should be able to extrapolate when it became really complicated ugly code.

I thought I would show you the YUV and gamma correction formulas.

Welcome to natural CCD saturation:) Being used to CMOS, the difference might be striking.

Hi John,

I find it strange that there is a difference between CCD and CMOS in regards to color saturation. If both CCD and CMOS are the same size and have the same color filter on them them the same amount of light will fall in each photon well.

Maybe you can't have the same filters for CCD and CMOS?

Cheers,
Take

They are not the same size at all! The Kodak pixel is 7.4um. The pixel on Red and SI and any low frame rate cmos is much smaller. The pixel on the SI sensor is 5um or something like that. The Kodak pixel is twice the area. The filters are also different depending on the manufacturer technology and experience. Kodak came from a huge film colorimetry background and Sony absolutely dominates the ccd market with its ccd technology. CMOS includes more processing on chip and comes with a higher noise floor. Sensor filters play a large part in saturation, the overlap, the relative balance etc.

I believe the cmos low saturation is inherent to the technology and the complexity of the sensor pixels. I have seen lots of unprocessed images from cmos sensors and this appears to be universally true.

Actually there are a number of CMOS manufacturers who get excellent color from their sensors . . . for instance, Micron can get the same level of color-saturation and accuracy from their camera-native RGB image, and so can Canon as what I'm seeing from the Kodak CCD's.

So I don't think it's fair to state that CCD=good color while CMOS=bad color. A lot of it has to-do with the manufacturing process, the pigments used, the color-fastness of the pigments (a trade-off of less saturation for more long-term robustness), and the compatibility of the color pigments with the manufacturing process.

Also the pixel size is 5um on the Altasens in order to get 1920x1080 in a 2/3" compatible format. And from seeing the work that Micron has done, small pixels (<5um) does not mean poor color saturation out-of-camera.

Thanks,

Jason

Yes, lots of tiny pixel CCD still get excellent saturation, tiny sensor multimegapixel still cameras etc.

But any time I tried to get low light saturation in a CMOS I had to process a lot. With CCD I think I should even reduce saturation in good light. The saturation is usually natural at mid levels.

I do believe a Canon 350d is poor in color performance compared to a D70s. But that's just personal preference.

Have you tried the Microns? The "low-light" performance of those sensors might not be as excellent as a large-sensor CCD, but the color saturation is very nice.

Another thing to realize is the CCD's are clock-constrained . . . for instance, if one wants to have a single camera that can be as "film-like" as far as is possible in the range of frame-rates that one can cover, you can't do that with CCD's at the moment.

Also CCD's can get very hot compared to a simliar CMOS, and the hotter they get, the noisier. They also use up a lot of power, which gets dissipated at some point along the line as heat. Various off-chip generated bias voltages, etc. also can cause issues, especially as the sensor head gets hotter and more current must be drawn.

So CCD's have their shortcoming as well.

A part from a Micron 2048xsomething CMOS frame I found on the web:

http://img107.imageshack.us/img107/7257/79093717ls7.jpg

It's just not realistic. This could be a bright red car and we will never find out.

On the Silicon Imaging site there is a page with LUT tables. There is a "no look" file and a sample:

http://www.siliconimaging.com/Digita...ds/no_look.zip

I don't believe the camera is desaturating on purpose, so this must be the out of camera saturation. What kind of processing is applied with a look file? Is there saturation processing?

Yes, CCD is harder to design, more expensive and problematic to get right with all the extra components and costs a lot more in development and materials. This is also true for most Italian supercars, still, many people will prefer one of those over a BMW with a equivalent engine:)

EDIT: I think the car analogy suits the situation. We all know that a top of the line BMW might be a better tool for most transport applications compared to something with italian engineering. But the italian car still has its market because many people like the sound of the engine, the engineering mentality, the way these things work and look. And even if the specifications might be similar, the italian car can certainly be a lot more engoyable and handle better in extreme scenarios, even though the engineering is much simpler, the technology is not as advanced and it doesn't come with 20 3-letter acronyms of its various systems/technologies. This type of car is a financial nightmare for any automotive company, but engineers and management know there are reasons to maintain the production.

Definitely . . . if you download the XML, there is a saturation matrix in there, and you can see all the settings that are being applied to the camera image.

In the end I feel that both technologies have their place, with advantages and disadvantages on either side . . . it's not just "marketing" false-hoods that have created the popularity around CMOS as you have described in your other posts. There are advantages, and ways to mitigate the disadvantages.

Choice is a good thing.

Increasing saturation with post look files does come at a cost though. It's better to get more from the camera directly so you avoid boosting noise etc. I made a comparison of the out of camera, neutral (looks undersaturated to me) and film look.

http://img178.imageshack.us/img178/1217/lookcompdz4.jpg

Yes, it does, but as noted, it's a "mitigated" loss, meaning that for a little more noise you get the "good" saturation we've been talking about, along with the benefits of flexible frame-rates, low-power, high temp tolerance, all data pipeline (on-board A/D converters), optical format compatibility with 2/3" and S16mm, up to 2K resolution, etc., etc.

Technology is always moving, and tomorrow's CMOS will make today's CCD's look bad and vice versa . . . both technologies will have their respective places for sometime as far as I can see.

There is one thing though that I am seeing, and that is a lot more R&D and intellectual property is being applied toward improved CMOS designs than what I'm seeing with CCD . . . I think a lot of this has to-do with the ability for "fabless" firms to design CMOS sensors compared to the difficulties required to create CCD's. As such, I think we will probably see CMOS in the long-run out-pacing CCD design, with the "end-results" being a bit of a "pseudo-CMOS/CCD sensor", that is CMOS designs being created on very high-end mixed-signal processes that are typical of CCD designs. At that point you'll get the advantages of both, with less of the disadvantages of either.

As you may have gathered I am not trying to make my camera have a certain look and I am taking a more scientific viewpoint. This is why I have taken so long to get the camera output perfectly linear (within 6%) and also get the colors as exact as possible as well.

This would allow the most consistent image in post and give you the most control over the colors.

I think one of the reasons that colors are pretty good already is because of linearity and getting black level correct. I've seen the same thing when calibrating a CRT projector and doing the greyscale tracking using a photosensor and a voltage meter instead of trying to do the same thing by eye.

Nope, you're right Take, and I'm sorry for hijacking your thread . . . I didn't want to get into a CCD vs. CMOS discussion, but just wanted to point out you've definitely done some very fine work here, and the images from your software look really nice.

I don't mind the thread hijacking.

Black level and contrast will boost the sense of saturation, just like with analog imaging systems. That's why this looks more saturated:

http://img265.imageshack.us/img265/9...hed1coplo0.jpg

I would set the CRT black by eye. The proper setting depends on room lighting, reflectance of surface and many other parameters. I wouldn't trust anything except the eye for this.

black level, yes you do that by eye, greyscale tracking should only be done using instruments.

Hi, so I've been working on calculating the color matrix from a picture of a ColorChecker.

Here is the result:
http://www.vosgames.nl/images/Mirage...cal_bp_col.png

I actually thought when I first saw the picture that it was too much, but then I actually looked at the ColorChecker under light and compared it to the screen and it is really close.

I guess you get used to the desaturated look after working on something a long while.

Now of course I also need to look at some real live pictures before I can truly say the picture is correct.

Cheers,
Take

So, I have shot some footage of the park from my window, to see how everything will hold up in real live.

I dropped it in Final Cut Pro and started editing it. I happy to say that FCP will run the timeline (edited footage) in real time when the image is zoomed to 50% or smaller, at 100% it will skip frames but still work pretty good. The footage is on the harddisk that I've recorded on, a single SATA western digital SA16 250 GB disk.

Because this is a third party codec it does not do real time rendering of things like transitions, but scrubbing through a transitions works pretty smooth.

In any case I am currently exporting the edit to my intermediate codec (so it renders in high quality). It says it needs two hours to render 1 minute and 35 seconds (I will need to solve some performance issues :-). I actually thought that FCP would render on both cores of my computer, but only 50% cpu time is used.

In any case I will try and put the footage online for all to see, does anyone have a preference for which codec I should use to export it? I thought I would use Apple's codec that they also use to encode their movie trailers in.

So it's 75:1 real time. What cpu is on your laptop? Any multithreading?

The noise looks very nice on the last sample.

I think the look is more important than precision. We know it's uncompressed video. WMV or MOV is ok.

Don't get the desaturated look syndrome:)

Hi there, it took a little longer because there was a bug in my codec stopping the last conversion. I didn't know about a certain data transfer method.

First I would like to apologise for the shaky image, I have no tripod mount made for my camera and was a bit excited being it my first real footage.

Anyhow, I finally made a conversion to h.264, which is the codec that Apple really likes. I did notice that when the image is particularly smooth and out of focus it tents to color band quite a bit, therefor I also added some still images to show the difference.

This was the workflow:
- Create calibration data for camera using DNFCalibrator (DNF stands for Digital Negative Format)
- Capture footage with Boom Recorder
- Drop footage on the Final Cut Pro timeline and edit (the timeline automatically reconfigures to the DNF Intermediate codec)
- Export footage as QuickTime with the DNF Intermediate codec
- Using the Compressor.app (Apple) convert the DNF codec to h.264 codec

You may want to right click and "save as" for the movie file, otherwise it will show inside the browser.

http://www.xs4all.nl/~takev/ThePark-H.264.mov
http://www.xs4all.nl/~takev/TheParkImage1.png
http://www.xs4all.nl/~takev/TheParkImage2.png
http://www.xs4all.nl/~takev/TheParkImage3.png

So, this footage was made with a CF16HA1 (16mm) which has a 43 degrees horizontal viewing angle. Which I think is a normal lens for this camera.
If I would add two other lenses, which should I order.

These lenses are available, third number is the 35 mm equivalent:
12.5 mm, 54 degrees, 35 mm
16 mm, 43 degrees, 45 mm
25 mm, 28 degrees, 70 mm
35 mm, 20 degrees, 100 mm
50 mm, 14 degrees, 140 mm
75 mm, 9 degrees, 200 mm

Take, congratulations, you're doing an amazing job! It's great to watch your progress.

The png show a lot of pixelation, is this a product of the de-bayering? There's some FPN too, is that right? If you look at the tree going across the yellow advertising there's a white outline in the blue channel. Im trying to work out whether this is a debayering artifact or whether it's the lens. (You can see chromatic aberations quite clearly, which has to be the lens - check the roof of the mini).

I would love to see some bright sunny sky and darker areas together sometime.

So the lens isn't so hot, i wonder what other choices there are for this sensor size? This is the fujinon right? This has been a concern for me and this particular sensor - getting a good lens on there. (unless of course the aberations are correctable?)

It also looks like you're shooting through a window as well? That can't be helping at all!

Does the camera has an OLPF? Im not sure it does, do you think this will become a problem when the camera is moving a lot?

The colours look very good and the range seems pretty good too. Very promising!

I'd like to see a wide lens the 12.5 because wide is more difficult than zoom. on 16mm a 40mm is usually the 'normal' lens so you're pretty much there already.

So when's the windows version come out ;)

cheers
paul

Thanks paul,

The pixelation is I think indeed from the debayer, maybe I should teach it how to do diagonal interpolation. Right now it seems to think, "this is neither horizontal nor vertical lets use the box interpolator".

I am thinking of an new design for a debayer, but it probably won't work, I would use a median filter because as they say, "median filters are edge preserving", I just have to see how true that is for demosaicing a bayer.

There is quite a lot of color aberation, it seems to be caused by the lens. I first thought it was my debayer algorithm, but the color shift was extending beyond the edges. Also in the video you can see the color aberation on a tree, the color shift swaps around when the tree is panned from the left to the right side of the sensor/lens.

Yes, I was shooting through the window, it is cold outside.

There is no OLPF, and as you see I am moving the camera a lot. I think the motion blur takes care of the aliasing. I am not sure what happens when I reduce the shutter time, I guess I will have to try.

There won't be a windows version, until Microsoft decides to implement a lot of APIs from Apple. It may be possible that at some point the QuickTime component will work on windows, so that at least it becomes possible to edit on windows.

Can you fix chromatic aberrations, by just looking at the distance from the center of the screen and shift red and blue back?

Sometimes, it depends on the lens and the type of abberation. the magenta/green type you're seeing changes depending on the contrast, so the top of the mini would show more. You can also on the tree trunk that when the lighter path behind crosses they are more obvious there. So it couldn't be a global shift. Point it through a tree at the sky, that'll show up the worst.

Is that fujinon designed for a single sensor or designed for a 3 CCD sensor? Do you have anything else to compare with?

http://www.vanwalree.com/optics.html is a nice reference.

A camera like the EX has CA correction built in (so it must be algorithmically possible for a given lens). You can see when zooming the lens that the aberations appear then disappear when you stop.

Shooting through the window really isn't helping your contrast in the scene.

The OLPF effect will most likely show more for textured materials when you're moving slowly around with a higher shutter speed. I suspect it might end up being an issue. Just slowly move the camera pointing at some patterned material perhaps and create a small region of interest uncompressed movie from it.

Have you seen any heat or thermal issues with the Pike?

Why did you choose the Pike in the end, did you look at some others first?

cheers
paul

Hello Paul,

I saw an article about fixing chromatic aberrations. It seems that is basically scales the color channels. You have to specify the amount on a per lens basis, so it will probably be a Final Cut Pro filter.

From what I understand de fujinon is a single CCD lens. I have an old Minolta 35mm reflex camera lens.

The Pike can become quite hot, I am planning to make a cooling system.

I have chosen the Pike because it was pretty much the only one that handles 1920x800 at high bit depth at 24 fps and IIDC that could be connected to an Apple and had drivers for it.

[QUOTE=Take Vos;815316]Hello Paul,
From what I understand de fujinon is a single CCD lens. I have an old Minolta 35mm reflex camera lens.
QUOTE]

I don't know ultimately what you intend to do with your system, but if you're hoping to use it out there in the field then the lenses are going to be key.

Ive been looking for suitable lens options for this sensor. It's between 2/3 and 35mm and there's quite a gap here. Some S16 lens *may* be big enough but for the most part the imaging circle will be too small as this sensor is bigger than S16. (it's also difficult to get technical details on a lot of older lenses)

You can stick 35mm on with a crop factor, around 2.5 i think. So a good 14mm lens works out to around 35mm fov, but the fastest i've found is f2.8 which'd be like having the DOF at f7 on 35. I think the better 35mm lenses would be fine resolution wise even though you're only using the centre.

There're a few lenses specifically for APS-C sized sensors, i have a canon 10-22 which is actually a pretty nice lens. But it's not manual so unless you have a birger mount it's not going to work. I don't know if there are any manual lenses for APS-C but they would be an interesting option because they're designed for the smaller sensor size and they'd be plenty big enough for this sensor. 10mm would be a nice wide fov on it. Again these tend not to be very fast.

So machine vision lenses are the most obvious choice. The fujinons look nice on paper but so far aren't performing too well in real world. Do you know what aperture you had that on - probably smaller than f1.4 i'd guess? The Pentax are probably at the same level and the scheider kreuznach are a lot more expensive, perhaps they'll perform better?

A lot of SLR lenses don't have much travel on the barrels for focus, i'm not sure about all the machine vision lenses.

So unless im missing something blindingly obvious (not the first time!), i think it might be a struggle to get some quality glass in front of the sensor to operate at reasonable fov at a flexible range of Dof for narrative purposes.

Conversely i don't think there's a suitable 1920x1080 CCD sensor in 2/3rds format -- which'd open up the choices enormously.

cheers
paul

Hello Paul,

All the 2/3 lenses I've seen are designed for 3CCD, not for a single sensor. 35mm SLR is interesting, especially if you need a long lens, but it is almost impossible to get really short 35 mm SLR lenses.
S16 lenses would also be interesting if we can find a couple that produce a 1" circle (altough at 2.40:1 ratio the circle may be slightly smaller).

Machine vision lenses remains the only solution if you want to buy them new. The Fujinon are not that expensive 250 USD per lens, the number of blades in the aperture is pretty good and the build seems very solid. The focus travel is pretty short but smooth, aperture setting is also smooth with a light click at each stop. I think you can find a follow focus that is geared for these short travel lenses.

I switched aperture quite a lot in the movie, it was quite dark outside, so I had it open pretty wide. I also wasn't that careful is focus.

When a dark image is shown my bad pixel detector basically disables all pixels and thus each pixel will be interpolated from its neighbour pixels, basically dropping in resolution.

This is not really a problem as it removes a lot of fixed pattern noise and smoothes out the image. But it will also show these blobs of noiseless dark patches which are a bit distracted compared to the rest of the image.

Luckily I already have measured the amount of white noise during calibration, so the only thing to do was to add a random amount of that white noise back onto the interpolated pixel. Now the patches are gone and it looks a bit more uniform.

Cheers,
Take

Take,

I hope im not polluting the thread (well i am) but this lens looks like it might be a good lens, the charts look pretty good and on a 1" should be better.

http://stilar.de/hp32638/Stilar_-2_8...bd6e8392XY25be

No idea how much though...

cheers
paul

Hi Paul,

This is not really pollution. I like to have a lively discussion on my thread.
Anyway, interesting lens, to bad there is only a single lens and not a whole series available, it is also very wide.

I think I've seen 1.2" lenses before, which could be interesting if I am ever going to the 2048 pixels wide sensor. But of course the alias filter for a 1.2" lens is probably wrong.

Cheers,
Take

Take

http://docter-optics.de/hp834/TEVIDON_-CCD-lenses-.htm

from the same people, so there are some others (of which only a couple apply to a 1")

When you say alias filter do you mean a OLPF? Does the Pike have something like that already? Does it have an IR filter? (you can get filters that do both)

cheers
paul

Paul,

Yes, I do mean OLPF when I say alias filter.

The IR filter for the pike is glued (or something) to the C-mount. The C-mount itself is mounted into a wider screw mount, with very narrow threads for back focus.

I do not see a way to add or change the build-in IR filter, but I may not have looked at it closely enough.