High Definition with Elphel model 333 camera

[Daniel Lipats]

Originally Posted by Noah Yuan-Vogel

whats with the color spots? what sort of debayer algorithm does the 333 use? i dont think i get that on my sumix m73 with sumix's lapacian debayer. I do with nearest color debayer though. what exactly does zmatte deartifact do? sharpness loss with filter doesnt look bad, but itd probably be nice not to have to filter and recompress all footage just to get rid of artifacts that should have been avoided with a good debayer in the first place.

Its possible its just reflected light from the sun, or the spotlight I had on the plates. The dishes are decorated in gold, so it would reflect a lot of light. Also its possible this formed from the color correction which was just "Auto Levels"

I'm really impressed by how much zmatte deartifact helped.

[Noah Yuan-Vogel]

I think those artifacts are specific to the debayer algorithm and its inability to correctly handle very small specular highlights which may end up saturating one color pixel but not others. Just a guess. Pretty sure its something that could be handled during debayering.

[Oscar Spierenburg]

http://s09.picshome.com/5fd/333framecczm.jpg

Does it work now?
Zmatte is used for chromakeying, but this is a part of the plugin to reduce color artifacts and jagged edges to get a good chromakey. It doesn't reduce the sharpness.
Maybe there are others as well.

[Steven Mingam]

Originally Posted by Noah Yuan-Vogel

I think those artifacts are specific to the debayer algorithm and its inability to correctly handle very small specular highlights which may end up saturating one color pixel but not others. Just a guess. Pretty sure its something that could be handled during debayering.

Yes it's definitly artifacts created by "simple" de-bayering algorithm, that's why it would be more interesting to do it in post :D

Here my quick test with bayer compression :

original
After separating the 4 planes and jpeg compression (q10 for G, q4 for R,B) Compression ratio was about 4:1.

You can find the raw files in the directory. I used photoshop 6 for de-bayering (with an old plugin) and jpeg compression (so max Q=12).

[Andrey Filippov]

Originally Posted by Steven Mingam

Yes it's definitly artifacts created by "simple" de-bayering algorithm

Yes, sure the algorithm now is too basic and we plan working on it. Unfortunately our application to http://code.google.com/soc/ did not work, but the project ideas are still there (including demosaic): http://wiki.elphel.com/index.php?tit...rithms_in_FPGA

I'm thinking to go ahead with students (on the same terms) and announce it a week or so after the student applications deadline at GSoC when some people will show up who missed the application time.

[Noah Yuan-Vogel]

I'm still interested to know how that video phil posted was so lacking in rolling shutter artifacts. On a similar note, ive been playing around with my sumix camera again (same micron sensor as the 333) and rolling shutter artifacting is hugely reduced by turning the camera 90degrees. I imagine this has been discussed before but its an interesting option for those interested in avoiding rolling shutter issues. rolling shutter is reduced in two ways by this: normal orientation causes artifacting on horizontal pans and rotated orientation replaces this with vertical pan artifacts which in my opinion are less common and less likely to be quick pans, and because the shorter lines are being exposed, the actual time difference between the exposures of adjacent lines is reduced because vertical resolution is (1.33, 1.77, 1.82, 2.4 or 2.66 times) lower than horizontal on normal video, so the artifacting is that many times less. of course this leaves you with a sensor with a width equivalent to a 1/3" sensor instead of 1/2".
anyway, i figure it might be worth throwing in the ability to record rotated images for people interested in minimizing the effects of rolling shutter. there are also plans to support kodak ccd's right? for which rolling shutter wouldnt be an issue, right?

[Wayne Morellini]

Originally Posted by Phil Stone

If its the mjpeg stream the data rate is going to be either dependent on the the jpeg size (how complex is the image??) , the camera cpu, the write speed of the hard drive (if you have a really cheap slow laptop drive) or perhaps the bandwidth.

So if your filming a face with blue sky the image will be small & thus also the data rate, If your filming in the forest with loads of leaves its going to be huge. You need to think about this before you set up the camera. There perhaps needs to be a variable % quality setting that adjusts the quality level of the jpeg stream?

You can see this effect with any digital camera. the elphel when filming mjpeg is in effect a super fast digital camera.

Phil

So, it does not maintain a constant data rate like most video formats? This is an problem because frame timing is the most important thing, not maintaining max data rate at the expense of timing.

[Wayne Morellini]

Originally Posted by Noah Yuan-Vogel

I'm still interested to know how that video phil posted was so lacking in rolling shutter artifacts. On a similar note, ive been playing around with my sumix camera again (same micron sensor as the 333) and rolling shutter artifacting is hugely reduced by turning the camera 90degrees. I imagine this has been discussed before but its an interesting option for those interested in avoiding rolling shutter issues.

Yeah, I bought this up on the digital cinema camera threads a few years ago, but the problem you should get is line flashing, as details move while lines are being scanned horizontally.

[Wayne Morellini]

Originally Posted by Steven Mingam

Here my quick test with bayer compression :

original
After separating the 4 planes and jpeg compression (q10 for G, q4 for R,B) Compression ratio was about 4:1.

You can find the raw files in the directory. I used photoshop 6 for de-bayering (with an old plugin) and jpeg compression (so max Q=12).

Whats the Q10, Q4 again, late can't remember back far enough?

An good attempt on the images. Had a close up of the pictures, and there is some color shift, a lot is slight and it looks natural. Some is like the debayered spectral highlights, and probably can be fixed up with the same software, but few of them. The resolution looks blurred, like on an cinema screen, but it does look similar to 4 times less resolution. Even in codecs like cineform, you still notice some of this, probably not as much, but still noticeable. I noticed some blocks in the lower right of the house, but did not notice much. This is probably not far off what an high end compressed ENG camera could achieve. Forgot to mention, the blur has virtually totally nuked the noise in the frame.

I would be curious how an grey-scaled bayer image debayers after going through an high 100% grey scale Jpeg compression.

[Andrey Filippov]

Originally Posted by Wayne Morellini

So, it does not maintain a constant data rate like most video formats? This is an problem because frame timing is the most important thing, not maintaining max data rate at the expense of timing.

It is possible to have the fps constant but it could be somewhat confusing in the current interface so we will make it easier to use.
Currently you can:
1 - limit fps to certain value (even if camera can run faster)
2 - limit maximal exposure (that autoexposure can set) to a certain value - if it is set equal or lower than the frame period - fps will not change.
We'll make that (with default settings -possible to disable) if a streamer is running autoexposure will be limited to a frame period.

[Steven Mingam]

Originally Posted by Wayne Morellini

Whats the Q10, Q4 again, late can't remember back far enough?

Q10 and Q4 is the quality of jpeg, photoshop 6 range from 0 to 12, so 4 is pretty bad (a lot of blocks) and Q10 is very nice (almost no artefacts). I included the after-compression raw files in the same directory, just browse it.

I would be curious how an grey-scaled bayer image debayers after going through an high 100% grey scale Jpeg compression.

I'll take a look, but for me, it looks very worthy to separate each plane for compression, because when debayering you'll do some interpolation which will filter some noise. So it' should be better to introduce noise before filtering than after :)

[Andrey Filippov]

Originally Posted by Steven Mingam

I'll take a look, but for me, it looks very worthy to separate each plane for compression, because when debayering you'll do some interpolation which will filter some noise. So it' should be better to introduce noise before filtering than after :)

It would definitely make sense to organize information in a wiki - this thread seems to be too big :-)
On page 33 I described how we dealt with compression of raw (before de-bayer) images in our model 323 cameras. In 333 this code might be broken but we'll have it again in 353/363

[Phil Stone]

Originally Posted by Wayne Morellini

So, it does not maintain a constant data rate like most video formats? This is an problem because frame timing is the most important thing, not maintaining max data rate at the expense of timing.

I think the ogg Mpeg4 version could be made to be a constant bit rate but most Mpeg4 streams like Xvid & DivX are variable between limits. With the mjpeg version it is what it says on the can, Jpegs! early versions of the camera would output jpeg frames into a folder via Mencoder. You then just imported these into Vegas or Vdub to convert into a avi that you could watch & edit. Every frame has a different size depending on your settings. You can adjust the settings so the quality level is low & that the camera holds a set frame rate. I pick a frame rate a little lower then max to give the camera some leeway if the scene becomes too complex.

Maybe the reason for the lack of rolling shutter effect is the frame rate is just 12fps. Its got time to take a full photo image like a normal digital camera. I think if you go a little faster to about 15-20fps you may see the effect. Then again Ive not noticed it with the 333 in any of the films Ive made. The 313 did when it was 1280x1024 & 15fps it was like a fluid effect. When the frame rate was increased to 22fps it was almost gone.

[Phil Stone]

http://www.tacx-video.com/Elphel/Elp...r-Belgium3.avi Another clip (right click & 'save destination') Its 43mb & has a little bit of the medieval cloth hall in my town & some cars.

[Wayne Morellini]

Originally Posted by Steven Mingam

Q10 and Q4 is the quality of jpeg, photoshop 6 range from 0 to 12, so 4 is pretty bad (a lot of blocks) and Q10 is very nice (almost no artefacts). I included the after-compression raw files in the same directory, just browse it.


I'll take a look, but for me, it looks very worthy to separate each plane for compression, because when debayering you'll do some interpolation which will filter some noise. So it' should be better to introduce noise before filtering than after :)

Yep, remember now, the Q4 is probably responsible for the detail loss on all the wood against blue sky. I think that an average even max Q for all colors will keep better image. Sacrificing on one or two colors means less precise interpolation for green in that position and vice a versa. I know we can't get 100mb/s at the moment, but 720p = around 192mb/s, which would be around 2:1 compression, if it was available, good to retain an lot of accuracy. But, around 3:1-4:1 is probably a satisfactory target. Maybe it could be systematically tested at 2:1, 3:1, 4:1, and 5:1, to see how it downgrades (with even Q across colors).