High Definition with Elphel model 333 camera

Oscar Spierenburg · January 5th, 2008, 08:02 PM

I had some great tests with the viewfinder. Very clear and bright image. Perhaps it'll even be good enough to focus most shots. I still have to put everything together though and make a robust housing around the tubes.

Daniel, what are your thoughts about an internet tablet and the Elphel+HDD? (skipping a Laptop/PC) I'm thinking about a combination of the optical viewfinder and 'only' a 2x or 4x focus aid on the tablet screen. (not necessarily at full frame rate)

Daniel Lipats · January 5th, 2008, 09:08 PM

The two ways to get a preview image from the 353 is to either use the imgserv which will provide a jpeg image of the last captured frame, or to use the real time RTSP stream.

The problem with both options is that they are going to be very high resolution images of 2000x800 for example. At the moment the camera has no ability to downres images for preview. This is overkill for small devices such as cell phones with lcd screens of ~64x64, PDAs 240x320, or internet tablets of ~640x480 - 800x400.

Not only does it become a problem to display such an image, but its a lot of data to process and the CPU capabilities of such devices are limited. Furthermore, when displaying the full frame most of the data is processed and disregarded.

One of the options I was considering is to have a device server. Clients such as a PDA, cell phone, internet tablet, laptop, or desktop pc would connect to this server. The server would then format the RTSP stream from the camera optimized for the connected device.

For example, the DOP may be using an internet tablet or a similar device to get a preview of the picture. He would turn the device on, it would connect to the server over WIFI and he would get a properly formated image of 800x600 for example. If he wants to zoom in for focus aid or detail all he has to do is use the touch screen, and the server would send a new, or reformatted video feed/image. Using this device he can also configure the camera and more.

Or here is another example. The visual effects supervisor would like to get an idea of how well a scene matches a miniature shot, key out the bluescreen, do a basic comp, overlay videos/images, or would just like to simply give the director an idea of things are coming together. Using the handheld device connected to the server he can do this by the touchscreen, it would be computed server side, and receive the realtime or image preview.

The possibilities are endless.

The disadvantage of using such a server is that its going to make the system more complex. Additional power will be required, and it would make the camera package bigger. But you did ask about my thoughts :)

It is not imposable to just use a high resolution image from 353 directly on the hand held device. Devices with wifi abilities are best suited and would take minimum work to implement. You could possibly get the imgserv jpeg at ~6 FPS. The RTSP stream requires a lot of cpu resources. I believe the recommended was 3ghz for optimal performance, so I don't think thats a realistic option. It is also possible that requesting an imgserv image at a high rate will take a toll on the camera CPU and lower the recording frame rate.

Andrey Filippov · January 5th, 2008, 11:29 PM

For the viewfinder mode it will be rather easy to make FPGA to provide 1/8 (or even 1/16) resolution (so 1/64 or 1/256 number of pixels) of uncompressed data in parallel to the video, virtually without using additional resources (so not slowing down the recording).

There is also another feature in the FPGA - not yet used (and needs some code to connect it to the output) - the amount of high-frequency components fro each of the 16x16 macroblock used in compression. This data can be used to provide info for the quality of focus if you average that data over selected region of the frame that you want to be in focus.

Oscar Spierenburg · January 6th, 2008, 09:47 AM

Quote:

Originally Posted by Andrey Filippov

This data can be used to provide info for the quality of focus if you average that data over selected region of the frame that you want to be in focus.

Andrey, ..this goes a bit above my head I guess. With data, do you mean in the form of a histogram or something?

What I meant with focus-aid, was only to show a the center of the image zoomed in 2x (like used in the ajax interface) and show that on the mobile device. The whole image would be unnecessary, because I would have it on the optical viewfinder. This is just an idea, because I understand that not everybody would be able to make the viewfinder... but than again, it would be a simple starting point.

Andrey Filippov · January 6th, 2008, 10:23 AM

Oscar,

It is more difficult to implement zoom window (w/o slowing down the main data) than to provide (per 16x16 tile) "focus quality" data, it is a by-product of the compression process, most of it is already implemented. Next time I'll get into FPGA code I can add output of such value over the specified window, we can display it as a bar in the UI.

Odd Nydren · January 6th, 2008, 05:12 PM

Quote:

Originally Posted by Andrey Filippov

For the viewfinder mode it will be rather easy to make FPGA to provide 1/8 (or even 1/16) resolution

Excellent!!! That would be VERY useful!

The resulting preview image would be perfect to ensure framing and make it very easy to control the camera using a wifi enabled device...like a cell phone with touch screen or an internet pad. Resulting in a very compact setup.

Quote:

Originally Posted by Andrey Filippov

There is also another feature in the FPGA - not yet used (and needs some code to connect it to the output) - the amount of high-frequency components fro each of the 16x16 macroblock used in compression.

Even better! I was hoping for a feature like that but thought it was impossible. (due to CPU power) This would be an excellent way to check image sharpness.

Quote:

Originally Posted by Andrey Filippov

This data can be used to provide info for the quality of focus if you average that data over selected region of the frame that you want to be in focus.

Exciting news!

If possible - please implement this! It would make it totally possible to not just control but also frame/focus the camera with a wifi device.

I'd love to build a PHP web page where I can switch between a 1/8 or 1/16 sized image stream for framing...and be able to switch to a "focus quality" image to check for sharpness. Add to that a few controls for exposure/timing etc and we have a really cheap and compact camera controller. very cool

//O.

Régine Weinberg · January 7th, 2008, 02:51 PM

great that would be heaven, will it be ready some days ??

Andrey Filippov · January 7th, 2008, 05:06 PM

Quote:

Originally Posted by Ronald Biese

great that would be heaven, will it be ready some days ??

Yes, sure. And focus quality in a specified window can be really soon.

What do you think - maybe this can be used to control the camera?
http://linuxdevices.com/news/NS2593777857.html

Oscar Spierenburg · January 11th, 2008, 10:08 AM

The optical viewfinder is almost ready to be mounted on the Elphel. I will make it detachable, so I can always adjust things. And I'll probably make another one without the image inversion, so I can see the image from the 35mm adapter right way up.

Andrey, I'm very exited about implementing the focus quality. How can it be used in the control-panel (how should you read the focus quality)?

Andrey Filippov · January 11th, 2008, 08:25 PM

Oscar, that will just be a value that will be updated after each frame compressed. The focus will quality will be controlled fro specified window (in 16x16 pixels increments.

It has some resemblance of the JPEG image size (but that depends on all the frame, not just the center area). If you make a series of images with different focus settings (all the rest being the same) you can compare image size - the larger, the better is the focus.

Matteo Pozzi · January 12th, 2008, 11:25 AM

cool the focus aid!
maybe ca be used the magnification area
1- position the magnification area in the point of the picture we want to focus
2- press the button focus aid
3- roteate the lens focus wheel to the right
4- press button focus aid
5- roteate the lens focus wheel to the left so that the software can calculate back the image size of the same (more or less) frame of the right rotation of the lens focus wheel and when the size of the frame is similar to the higher achieved a green indicator on the gui will tell you!

nice work oscar!

Oscar Spierenburg · January 12th, 2008, 02:38 PM

The viewfinder is about ready and working great. I haven't had time to put the sensor and main board back in the camera body, but that'll be ready tomorrow.

Odd Nydren · January 14th, 2008, 03:19 AM

Quote:

Originally Posted by Andrey Filippov

Yes, sure. And focus quality in a specified window can be really soon.

What do you think - maybe this can be used to control the camera?
http://linuxdevices.com/news/NS2593777857.html

That would be awesome!
..might be tricky though as it doesent have wifi...? ..or maybe there's a way to hack it and use the USB port?

Maybe this one would be easier (although not linux based :( )
http://www.youtube.com/watch?v=lbdmM...eature=related
Its got opera, wifi and fast mpeg-2 playback.

Err - I just realized - archos is linux based and uses Qtopia... http://trolltech.com/products/qtopia.../qtopiadevices

Odd Nydren · January 14th, 2008, 03:22 AM

Hi all,

I made a little framerate calculator...and thought you guys might find it useful too. You can enter the sensor size and full frame framerate and it will give you an approximation of the framerate att smaller image capture sizes.

Please let me know what you think.
If you need any other features - ask and I might put it in.

http://www.oddpixel.com/fps.html

//O.

Matteo Pozzi · January 15th, 2008, 03:46 AM

Quote:

Originally Posted by Matteo Pozzi

cool the focus aid!
maybe ca be used the magnification area
1- position the magnification area in the point of the picture we want to focus
2- press the button focus aid
3- roteate the lens focus wheel to the right
4- press button focus aid
5- roteate the lens focus wheel to the left so that the software can calculate back the image size of the same (more or less) frame of the right rotation of the lens focus wheel and when the size of the frame is similar to the higher achieved a green indicator on the gui will tell you!

nice work oscar!

or better :
a upper green arrow if the current frame is bigger than the previous 2 and a red down arrow beside the magnification 2x, 4x square if the current frame is smaller!
:-)

January 7th, 2008, 02:51 PM	#862
Régine Weinberg Major Player Join Date: Jan 2004 Location: Bordeaux, going to Bangkok, 2011 Posts: 232	Oh wonderfull great that would be heaven, will it be ready some days ??

January 11th, 2008, 10:08 AM	#864
Oscar Spierenburg Major Player Join Date: Jan 2005 Location: (The Netherlands - Belgium) Posts: 735	Viewfinder update... The optical viewfinder is almost ready to be mounted on the Elphel. I will make it detachable, so I can always adjust things. And I'll probably make another one without the image inversion, so I can see the image from the 35mm adapter right way up. Andrey, I'm very exited about implementing the focus quality. How can it be used in the control-panel (how should you read the focus quality)? Attached Thumbnails

January 12th, 2008, 02:38 PM	#867
Oscar Spierenburg Major Player Join Date: Jan 2005 Location: (The Netherlands - Belgium) Posts: 735	The viewfinder is about ready and working great. I haven't had time to put the sensor and main board back in the camera body, but that'll be ready tomorrow. Attached Thumbnails

January 14th, 2008, 03:22 AM	#869
Odd Nydren Regular Crew Join Date: Jan 2007 Location: London, UK Posts: 88	Size & framerate... Hi all, I made a little framerate calculator...and thought you guys might find it useful too. You can enter the sensor size and full frame framerate and it will give you an approximation of the framerate att smaller image capture sizes. Please let me know what you think. If you need any other features - ask and I might put it in. http://www.oddpixel.com/fps.html //O.

January 5th, 2008, 08:02 PM	#856
Oscar Spierenburg Major Player Join Date: Jan 2005 Location: (The Netherlands - Belgium) Posts: 735	I had some great tests with the viewfinder. Very clear and bright image. Perhaps it'll even be good enough to focus most shots. I still have to put everything together though and make a robust housing around the tubes. Daniel, what are your thoughts about an internet tablet and the Elphel+HDD? (skipping a Laptop/PC) I'm thinking about a combination of the optical viewfinder and 'only' a 2x or 4x focus aid on the tablet screen. (not necessarily at full frame rate)

January 5th, 2008, 09:08 PM	#857
Daniel Lipats Major Player Join Date: Feb 2006 Posts: 260	The two ways to get a preview image from the 353 is to either use the imgserv which will provide a jpeg image of the last captured frame, or to use the real time RTSP stream. The problem with both options is that they are going to be very high resolution images of 2000x800 for example. At the moment the camera has no ability to downres images for preview. This is overkill for small devices such as cell phones with lcd screens of ~64x64, PDAs 240x320, or internet tablets of ~640x480 - 800x400. Not only does it become a problem to display such an image, but its a lot of data to process and the CPU capabilities of such devices are limited. Furthermore, when displaying the full frame most of the data is processed and disregarded. One of the options I was considering is to have a device server. Clients such as a PDA, cell phone, internet tablet, laptop, or desktop pc would connect to this server. The server would then format the RTSP stream from the camera optimized for the connected device. For example, the DOP may be using an internet tablet or a similar device to get a preview of the picture. He would turn the device on, it would connect to the server over WIFI and he would get a properly formated image of 800x600 for example. If he wants to zoom in for focus aid or detail all he has to do is use the touch screen, and the server would send a new, or reformatted video feed/image. Using this device he can also configure the camera and more. Or here is another example. The visual effects supervisor would like to get an idea of how well a scene matches a miniature shot, key out the bluescreen, do a basic comp, overlay videos/images, or would just like to simply give the director an idea of things are coming together. Using the handheld device connected to the server he can do this by the touchscreen, it would be computed server side, and receive the realtime or image preview. The possibilities are endless. The disadvantage of using such a server is that its going to make the system more complex. Additional power will be required, and it would make the camera package bigger. But you did ask about my thoughts :) It is not imposable to just use a high resolution image from 353 directly on the hand held device. Devices with wifi abilities are best suited and would take minimum work to implement. You could possibly get the imgserv jpeg at ~6 FPS. The RTSP stream requires a lot of cpu resources. I believe the recommended was 3ghz for optimal performance, so I don't think thats a realistic option. It is also possible that requesting an imgserv image at a high rate will take a toll on the camera CPU and lower the recording frame rate.

January 5th, 2008, 11:29 PM	#858
Andrey Filippov Major Player Join Date: Apr 2006 Location: Magna, Utah Posts: 215	For the viewfinder mode it will be rather easy to make FPGA to provide 1/8 (or even 1/16) resolution (so 1/64 or 1/256 number of pixels) of uncompressed data in parallel to the video, virtually without using additional resources (so not slowing down the recording). There is also another feature in the FPGA - not yet used (and needs some code to connect it to the output) - the amount of high-frequency components fro each of the 16x16 macroblock used in compression. This data can be used to provide info for the quality of focus if you average that data over selected region of the frame that you want to be in focus.

January 6th, 2008, 10:23 AM	#860
Andrey Filippov Major Player Join Date: Apr 2006 Location: Magna, Utah Posts: 215	Oscar, It is more difficult to implement zoom window (w/o slowing down the main data) than to provide (per 16x16 tile) "focus quality" data, it is a by-product of the compression process, most of it is already implemented. Next time I'll get into FPGA code I can add output of such value over the specified window, we can display it as a bar in the UI.

January 11th, 2008, 08:25 PM	#865
Andrey Filippov Major Player Join Date: Apr 2006 Location: Magna, Utah Posts: 215	Oscar, that will just be a value that will be updated after each frame compressed. The focus will quality will be controlled fro specified window (in 16x16 pixels increments. It has some resemblance of the JPEG image size (but that depends on all the frame, not just the center area). If you make a series of images with different focus settings (all the rest being the same) you can compare image size - the larger, the better is the focus.

January 12th, 2008, 11:25 AM	#866
Matteo Pozzi Regular Crew Join Date: Jul 2005 Location: Milan, Italy Posts: 169	cool the focus aid! maybe ca be used the magnification area 1- position the magnification area in the point of the picture we want to focus 2- press the button focus aid 3- roteate the lens focus wheel to the right 4- press button focus aid 5- roteate the lens focus wheel to the left so that the software can calculate back the image size of the same (more or less) frame of the right rotation of the lens focus wheel and when the size of the frame is similar to the higher achieved a green indicator on the gui will tell you! nice work oscar!