Have a look at light field technology it's going to change the way we take photos! Pretty amazing stuff...
Lytro's light field camera captures 'unprecedented' images, lets you choose focus later -- Engadget (http://www.engadget.com/2011/06/22/light-field-camera-captures-unprecedented-images-lets-you-cho/)

Wow, finally coming to the consumer market. I've been wondering when this would hit since I first read about it in 2005.

Light Field Photography with a Hand-Held Plenoptic Camera (http://graphics.stanford.edu/papers/lfcamera/)

Can't wait to see the model that ships & even more can't wait until they get it into video cameras. I think this could even be great for green screening w/o a green screen. Just tell the software that everything more than "X" number of feet away, delete.

This has the potential of making the "DSLR revolution" a forgotten memory. Who wants shallow DOF recorded, when you can have any DOF in post?

I wonder if 2 different depth ranges could be in focus, while the areas in front, behind and between are out of focus.

Dang, that really IS a game changer...

As for the game changer, I think all the cameras they have for this are pretty low resolution, like 1-2mega pixels (I could be wrong). Not sure if it has to do with the lens array, or the amount of information that has to get passed through.

If it is the lens array, then making this for video seems not too far away. If it is the amount of information that keeps the resolution small, then video would be farther in the future, as they'd have to get the frame rate up.

Still the amount of R&D in cameras today & how quickly they're improving & getting cheaper, nothing is too far away to wait for.

I wonder if 2 different depth ranges could be in focus, while the areas in front, behind and between are out of focus.

I think this would be possible, as it would probably just making a good algorithm in the software to handle it.

I think that it would be great to adjust an over the should shot 2-shot, so the background stays locked at a constant out of focus; subject stays locked in focus; but the interviewer's focus can be adjusted.


I can't wait to get my hands on one of these cameras & start playing around with it. Game changer I don't think is the right phrase, it seems bigger than that if it can do half the things I'm thinking about.

Sorry to be a wet blanket, but you rarely get something for nothing - and that is the case here.

The principle has been around for a while - Plenoptic camera - Wikipedia, the free encyclopedia (http://en.wikipedia.org/wiki/Plenoptic_camera) - and whilst perfectly sound, the flexibility (such as being able to refocus) comes at a price. In this case, it's fundamentally that for a given image resolution, the data etc corresponds to a far higher resolution image. From the wikipedia link:
The Adobe light-field camera is a prototype 100-megapixel camera that takes a three-dimensional photo of the scene in focus using 19 uniquely configured lenses. Each lens will take a 5.2 megapixel photo of the entire scene around the camera and each image can be focused later in any way.

Hence, for the foreseeable future, even with a high resolution base (and big file sizes), the output images are likely to be quite low-res by todays standards, and don't expect video at decent resolutions with current technology.

Read the info on their web site - it appears that some fancy mathematics have been applied to reduce the hardware significantly, and their samples don't look too bad... They specifically refer to reducing a roomful of cameras to a pocketable device, and they apparently have photographers "in the field" working with these right now...

Too early to know much, but it certainly could represent a breakthrough technology presuming resolution is within expectations. They do claim to be shipping in the relatively near future.

As for the amount of data, there have been several algorithms I've seen suggested that might drastically reduce the amount of data required to achive acceptable resolution - offhand, I can see several of the "academic" anouncements of experimental techniques I've heard of coming together to make this "work".

It will be interesting to see whether video is on their radar - at the moment it appears stills are the "focus", but it would be quite interesting to apply the principles to video... if they can apply the concepts across time instead of in a limited timeslice, it should be feasible, if not now, in the future with faster more capable DSPs...

Read the info on their web site - it appears that some fancy mathematics have been applied to reduce the hardware significantly, ........... Too early to know much, but it certainly could represent a breakthrough technology presuming resolution is within expectations.
As a lot of replies on their blog say, the site is very lacking on specs - and noticeably to do with resolution, or even whether zoom lenses would be possible. I'm not doubting that it will work - but strongly suspect that we may be looking at something geared far more towards the consumer point-and-shoot market than serious photography.

Which is exactly where the technology is likely to be most relevant. Take a 16 megapixel pixel image, and derive something like a 1 megapixel refocussable image (with very large file size), maybe?
They specifically refer to reducing a roomful of cameras to a pocketable device,
Which again comes at a cost. The "roomful of cameras" refers to being able to produce a result whereby you can gain an effect of walking round (and over, and below) a scene, as well gaining a conventional 3D image, or a 2D iamge with variable depth of field - depending how the data is processed. Such techniques are not new - for example, a museum may use them to record a statue, say, in unprecedented detail.

This company is likely talking about a product with very limited view angle etc by comparison - but which may be just what some customers want..

The clue may be in the FAQs of the company:
What began in a lab at Stanford University has transformed into a world-class company, forty-four people strong, sparkling with talent, energy and inspiration.
Wikipedia refers to the Stanford research -
A team at Stanford University used a 16 megapixel camera with a 90,000-microlens array (meaning that each microlens covers about 175 pixels, and the final resolution is 90 kilopixels) to demonstrate that pictures can be refocused after they are taken.
Now I don't doubt that this has progressed in the six years since then, but there are good technical reasons why the final product is far more likely to be something intended for relatively low-res point and shoot - not what most on this forum would call serious photography. But I wait to see more detailed information on their website.

In the meantime, you may be interested in the original Stanford University research paper from 2005 - http://graphics.stanford.edu/papers/lfcamera/lfcamera-150dpi.pdf - one of the authors is the founder and CEO now listed for Lytro.

Taking this to the extreme...

Imagine a big, empty cube that you would bring into a room. It would have Light Field technology, HDR, RAW recording, and virtual camera positioning. Turn it on, hit REC, and yell, "action." The focus, exposure, color balance, and camera position/movement could all be chosen in post.

Some would say that this takes away from the art of filmmaking. I would disagree. It would decrease the skills needed to shoot, but could allow more artistic end results. It would also save a ton of time. There would be no more botched focus, exposure, colors or unintentionally jerky motions. We wouldn't have to move and level tracks, change lenses, and on and on. It would ONLY be about the performance of the actors.

This would be killer for composites as well. Camera motion tracking between layers would be perfect.

But for now, I'll focus, adjust the camera settings, and deal with dollies and jibs and imperfect movements. And I'll keep saying "cut! We need to do that again!"

David Heath, remember the Nikon D1? A "professional" camera with 2.7 megapixels, hardly usable resolution for any "professional" still photo.

But how long did it take after that to get DSLR cameras with respectable resolutions?

If I can get a 1megapixel light field camera this year for under $300, I'd go crazy & start playing around with it as much as I can. It would be a novelty at that resolution & not great for paying jobs, but still fun to play with.

And if it sells well enough, you can bet that all the major camera manufactures will be releasing better cameras as soon as they get the chance to take the money from buyers. (I'd be shocked if they weren't already doing R&D).

David Heath, remember the Nikon D1? A "professional" camera with 2.7 megapixels, hardly usable resolution for any "professional" still photo.

But how long did it take after that to get DSLR cameras with respectable resolutions?
Yes, that's a good point. The point about the D1 was that it was targeted primarily at photojournalists, and for news the resolution was "good enough", but the time saving by no processing, and being instantly able to send the results electronically was key. But what the Stanford team produced was 0.09 megapixel output resolution (292x292), and that from a starting point of 16 megapixel. Tricks can obviously be done by trading off refocussing coarseness for resolution, but I think even 1 megapixel output resolution may be highly optimistic. We'll see, but something like 640x480 (0.3 megapixel) max might be more likely - and more than good enough for "point and shoot" and display on a smartphone screen.

The 16/0.09 is a ratio of nearly 200:1. That means to get an output resolution of even 2.7 megapixel would mean a sensor of about 500 megapixel - and video would mean reading and processing it 60 times each second. Never say never, but I don't see it happening soon.
[/QUOTE]If I can get a 1megapixel light field camera this year for under $300, I'd go crazy & start playing around with it as much as I can. It would be a novelty at that resolution & not great for paying jobs, but still fun to play with.[/QUOTE]
I don't disagree with any of that, just don't expect too much, too soon.

David -

Think on this for a moment. If I have grasped the fundamentals properly... the design uses MULTIPLE lenses with multiple "views" of an object from ever so slightly different angles... meaning there is an ABUNDANCE of data sets, which could easily be interpolated into a MUCH higher resolution image with some fancy math.

I recall an article (think it was linked here?) where they were using some "fancy math" to take relatively small file sizes and interpolate the details (something which still baffles me, as I don't see how you could get something from "nothing"!? Here there IS something though!) in order to recreate very high resolution, high detail images.

So let's take those two "technologies", mush them together, take a large number of ever so slightly different data sets, combine with algorithms that would analyze all those data sets at a high rate of speed, consider all the similarities and differences and "compute" a resulting image (looks sort of like what they are hinting at). You're not limited by "film", or sensor resolution, your only limits are data throughput and the efficiency of your algorithms in handling the data, and perhaps the quality of your lens(es).

In effect you get infinite focus (check), nearly infinite exposure (they claim very good low light), and nearly infinite theoretical resolution... gives me a headache thinking about it, and the computing horsepower in that camera would be pretty impressive, BUT methinks that's how they are doing it.

From a video standpoint, it may become rather problematic simply because scaling the calculations and horsepower required to take a single image enough to handle 30-60 images per second... well, it is mind boggling... but give it time, it's easy enough to do if the foundation is there, just have to wait for computing horsepower to catch up!

The more I contemplate this technology, the more intrigued I get - there is not a huge leap from this to "holocameras" capable of complete 3D capture, and all sorts of other crazy stuff!! Even crazier, what if you could "reverse" the algorithms to make a holo projector!? Eh, I'll take the camera since it looks like it's closed to "baked" already!


FWIW, the above is my "guess" based on pretty limited review of their website and that old technical paper, I could be completely off base, so don't take it as gospel or anything beyond a WAG (Wild A** Guess) at what is coming... but it's fun to contemplate!

The 16/0.09 is a ratio of nearly 200:1. That means to get an output resolution of even 2.7 megapixel would mean a sensor of about 500 megapixel - and video would mean reading and processing it 60 times each second. Never say never, but I don't see it happening soon.

Except that it is a little hard to define what would be meant as video processing, since much of the image processing with LFT is done in post. You could imagine shooting with just a huge amount of high speed storage, and then doing all the video processing in post.

All of these plenoptic cameras can be seen as a hack to do bokeh in software rather than
just using a wider aperture. The bonus of simulating a wide aperture in software is that you
can pick the focus plane after the fact.

It helps to understand where bokeh comes from but I think you can do a thought experiment
and get an idea for how this all works. I used the following explanation on
another website but maybe it will work better here:

Imagine you have a stereo rig with reasonably normal lenses that are almost zero
radial distortion and have the left eye and right eye files on two separate tracks in
your favorite video editor. Set the track on top to be 50% opaque so that your
output video is just both frames arithmetically averaged together.

In this new output video, most things will have two ghosts. Say you shift the
top frame left a few pixels until some of the objects loose their ghosts and
line up. You will probably find that the objects that it are possible to line up
(remove the doubled ghosts) are the same distance from the camera,
Congratulations, you have just focused your plenoptic camera at a particular depth.

You will notice though that everything in front of and behind the plane that is
in focus has two ghosts. This "blur" is basically the bokeh of your system. It
arises from the fact that the left edge of your new aperture views reality from
a different angle than the right edge of your aperture (the right eye camera).

Now this bokeh sucks. It's not a nice round diffuse ball, it is two points that tend
to make everything just look ghosted rather than actually blurred.

If you want a nice bokeh, use 20 cameras arranged on some sort of regular
two dimensional grid. It's more difficult to figure out how to translate each
view left and right so that the depth plane that you want is in focus, but it is
doable. Hard to think about but hopefully this makes sense.

If you want a whole lot of bokeh, add cameras that are farther from
the middle of the grid. If the bokeh is ugly (uneven/point like) add more cameras
between the cameras already on the grid.

You could also just make one big lens that was the size of the
frame holding the grid of cameras and get the same bokeh, but this could
be heavy and unwieldly. But the reason that the big aperture has big bokeh is
because the edges of the aperture are far apart they see different views of the
scene that don't line up right. Hence when they get averaged together, they
come out blurry and turned into bokeh.

One note:


meaning there is an ABUNDANCE of data sets, which could easily be interpolated into a MUCH higher resolution image with some fancy math.

No. These cameras have sacrificed spatial resolution in exchange for recording
angular information about how a light ray came into the camera. You can not generate
a high resolution 2 dimensional image. What you can do is generate a low resolution
3 dimensional image (or a surface mesh) more easily that you can with a stereo camera.

David -

Think on this for a moment. If I have grasped the fundamentals properly... the design uses MULTIPLE lenses with multiple "views" of an object from ever so slightly different angles... meaning there is an ABUNDANCE of data sets, which could easily be interpolated into a MUCH higher resolution image with some fancy math.
No, it doesn't work like that. It uses multiple lenses in front of the image plane to take multiple views, indeed, but they are all low resolution. The maths won't work to then reconstruct a high res image, it will work to reconstruct the effect of varying focus.
In effect you get infinite focus (check), nearly infinite exposure (they claim very good low light), and nearly infinite theoretical resolution... gives me a headache thinking about it, and the computing horsepower in that camera would be pretty impressive, BUT methinks that's how they are doing it.
No - you get the ability to alter focus in post, but only over a finite range - here fixed by the number of sub images. Infinite exposure is not true - what you will be able to do is correlate the images to improve s/n, but only to a limit. I suspect it will be of the order (in the Stanford case) of what would be achievable if they had simply had a 270x270 sensor of the dimensions of the 16 megapixel one they used. (Hence each photosite being large.) And the resolution will be far from infinite, it will be equivalent in this design to the number of microlenses, and the number of pixels under each microlens will determine the extent of refocussing that can be done.

All this is illustrated very clearly in Figure 13 of the Stanford report, and the text beneath it.

If you doubt me, try e-mailing the company and asking directly what the max output resolution of the system for sale will be?
You could imagine shooting with just a huge amount of high speed storage, and then doing all the video processing in post.
Even reading all the data off a 500 megapixel sensor at 60x times a second is a mammoth task, and, I suspect, way, way beyond current practical technology, and that's before we even start to think about processing. Applying this to high definition video may be theoretically possible - but practically is likely to be way, way off.

[EDIT - Just seen the last paragraph of Dustins reply above - exactly so, well put.]

All of these plenoptic cameras can be seen as a hack to do bokeh in software rather than just using a wider aperture.

It's more than just wide-aperture bokeh. It's the ability to pull focus perfectly every time in post.

Try a shot with a close focus lens and a dolly pulling back from an extreme closeup. Just try to maintain critical focus. After 20 to 30 takes, you'll re-design the shot. But with a plenoptic solution, you can nail the shot in one take. That's huge.

Of course, the technology isn't there yet, and may not be for decades, but never buzzing focus again has a lot of practical value.

It's now over six months since this was previously discussed, and I thought at the time the product was due to ship soon? It's still given as "Pre-order here" on the website - has anybody heard anything more? Or seen a prototype in the flesh?

http://www.dvinfo.net/forum/digital-video-industry-news/501795-first-commercially-available-light-field-camera.html

Yes, but the last post in that thread (by me) is even longer ago (about 9 months). The product was supposed to hit the shops well before Christmas - but where is it? The manufacturers site still seems to have it as "pre-order" only. Anybody heard anything recent?

I think they shipped a batch out to "early adopters" on their waiting list - they only had MAC software, so I'd suspect they are waiting to have Windows software available for wider release. They post regularly on Facebook, and have been doing lots of "PR" and public showings, so I'd guess it's "coming".

I was surprised at how quickly they announced they would ship, maybe they had supply chain issues like many other companies last year? Still be faster than many products from announcement to release...

This may have been posted in the past, but here's a really cool website to this new still camera. It shoots everythng in focus, then you can change the focal points afterward. The link will get you to some samples. It's pretty cool to actually change the focus in the pictures real time.
I know it's not a video camera, but maybe this technology can transfer to video soon.

https://www.lytro.com/living-pictures#living-pictures/1695?&_suid=829

Here: http://www.dvinfo.net/forum/digital-video-industry-news/497512-light-field-technology.html

And here: http://www.dvinfo.net/forum/digital-video-industry-news/501795-first-commercially-available-light-field-camera.html


When I saw the new thread, I was really hoping that someone at DVInfo had purchased one of these & started taking photos. I was so close on purchasing one, just didn't have the extra cash to justify it. If anyone does have one, please share.

And - as I've said in each of the previous threads - whilst what is claimed for it is perfectly true, it's what doesn't get said that is most relevant.

And that is the resolution of the final images that can be output from it. You trade off resolution for refocussing ability. It should be fine for images on the web, but not serious photography, and lytro have been very reluctant to discuss resolution.

The technology is already having industrial applications via other companies where depth information is more important than absolute resolution, such as in things like face recognition. (eg See Applications - Raytrix GmbH (http://www.raytrix.de/index.php/applications.html) ) Don't expect the technology to transfer to video anytime soon. At normal frame rates, you'd be talking about a massive data rate, only to end up with a standard definition image - albeit one that can be refocussed.

Lytro has certainly had some great marketing these last few months.

Maybe if they tie up with a certain company whose name starts with an 'A', so they could beat a certain rival who has just released a '41'MP camera phone...who knows?

They need to make a 1080p capable lytro and a plugin for after effects for focus control keyframing, then I might bite. Resolution isn't much for photo, but for incorporating lytro stills into video/motion graphics it could be a creatively useful tool.

Be careful what you wish for, Nevin! Wouldn't want to be the guy sitting on a station tracking focus on a moving shot in Transformers XII.

Digital photography magazine has an excellent in-depth review of this camera on their website, which is something like dpreview or dpreviews dot com. This camera is really only a toy at this point, but may be the start of something useful. The camera is enclosed in a 1.5 by 1.5 inch rectangular shell which is 4.4 inches long, and captures 1080 by 1080 pixel equivalent (approx 1.2 Mpixel) photos. The software, at last reading, worked only on Mac computers, and it would appear that the company hopes to sell out to Apple in the near future. The data files for the 1.2 Mpixel photo are large, and upscaling to a useful photo size would require very large data files. Interesting, but not yet ready for prime time.

Thanks Mark. Well worth reading, and bears out much of what I've said - the refocussing ability comes primarily at the expense of resolution.

The article raises a point that I hadn't really thought about - what they call "refocussability" or the range over which the image can be refocussed. They refer to it as equivalent to depth of field in conventional photography, but instead of the depth within with everything is IN focus, it's the depth within with everything CAN be refocussed.

Point is it's not infinite, and (as with dof) the more you zoom in, the less it is. Hence, much of the time the camera still does need to focus to an extent more or less conventionally at the time of shooting.

The other point is that the refocussing "smoothness" is governed by the no of microlenses. More microlenses, smoother the refocussing abilities - but the lower the output resolution. So the "11 megarays" COULD be used to give a final 5.5 megapixel image - but then likely only be able to be refocussed over two planes! Point is that "refocussability" could give you the ability to focus just in front of or just behind the plane you really want - but possibly not actually on it! Of course, more microlenses equals more fineness, but then expect the resolution to drop away.

If this camera is "11megarays" and they quote 1.2 megapixels as final output resolution, it indicates strongly that the microlenses are in groups of 3x3. That similarly indicates that you can expect to move focus between 9 separate planes of focus in the post stage.

Digital photography magazine has an excellent in-depth review of this camera on their website, which is something like dpreview or dpreviews dot com. This camera is really only a toy at this point, but may be the start of something useful. The camera is enclosed in a 1.5 by 1.5 inch rectangular shell which is 4.4 inches long, and captures 1080 by 1080 pixel equivalent (approx 1.2 Mpixel) photos. The software, at last reading, worked only on Mac computers, and it would appear that the company hopes to sell out to Apple in the near future. The data files for the 1.2 Mpixel photo are large, and upscaling to a useful photo size would require very large data files. Interesting, but not yet ready for prime time.

so it is already HD at 1080x1080, it could still have it's use in video production if we can control the focus over time on a timeline...

so it is already HD at 1080x1080, it could still have it's use in video production if we can control the focus over time on a timeline...
In a video sized frame (16:9) you're talking about the output equivalent to 1280x720 - and the implcations are that that the actual resolution is likely to be less. So 720 HD maybe, but not 1080.

But that's for stills. (And for a relatively coarse refocussing ability.) I don't think there's any suggestion yet that video is feasible. It's one thing to take the 16 "megarays" off the chip and record it as a still photograph - totally another to do that 24 times a second (let alone 50/60).

And even if it was possible, the ability is likely to come at the expense of other factors. The basic photosite size is small, and that's likely to affect dynamic range, for example.

Sorry, It's just not realistic, never say never, but I don't expect to see it in my lifetime, if ever.

I'm not talking about recording video. I mean animating still pictures in after effects for effect in a video production. For title/full-screen supers, diagram purposes, etc.

Say for a concert promo video/commercial. I can see using concert pictures taken with lytro, animating these still photos by rack focusing through thousands of people stopped in 1 moment in time.

You could do that now by taking an image with a very deep DoF and then selectively de-focussing the image to simulate out of focus or shallow DoF.

Lytro is still an interesting technology, but the flaw is that the camera is still using a lens. If there is a lens, then it's bringing light in to focus at certain points or over a certain range. I suspect the technology that will truly give a focus anywhere after capture ability won't use a lens as we know it.

I can see using concert pictures taken with lytro, animating these still photos by rack focusing through thousands of people stopped in 1 moment in time.
Sorry for misunderstanding - I can see what you mean, I suppose a variation on the theme of tracking round a "frozen in time object".

It's an interesting thought, but I'm still not sure the lytro technology is even up to that, not for quality work anyway. You'd have to check it in practice, but I suspect it won't be quite high enough resolution, and give a steppiness to the refocussing. So with (say) 100 rows of seats the effect would be more like being able to have focus on every fifth row and mixing between these planes of focus - not a smooth rack. (Imagine having 16 cameras all focussed at different planes - that's equivalent to the raw material you have to work with.) I would be interested to see it in practice.

I'd think they could incorporate this ability into dSLRs in a manner akin to exposure bracketing. Would be nice to tweak focus after the fact when doing a timelapse. Canon???

Mark

Lytro Camera Produces Amazing, Interactive Pictures (PHOTOS) (http://www.huffingtonpost.com/2012/08/27/lytro-camera-pictures_n_1827975.html)

Interesting.


CS

Sort of, I guess.

I had one on pre-order, got it this spring. Tried it out for a day or two and sent it back.

It's an interesting concept and as the capabilities progress (3D) along with the resolution, it will likely be more formidable. For now, I found the image quality lacking (reminded me of the stills mode found in camcorders five or six years ago--not so good) and the ergonomics funky (novel and attractive package, but the touch sensitive zoom control was erratic).

I'll keep an eye on them but I can't recommend the first gen product; it's kind of a one trick pony. You go walking around searching for things to shoot on extreme planes to take advantage of the depth of field thing, which isn't really photography, more like a novelty.

My opinion of course.

It's an interesting concept and as the capabilities progress (3D) along with the resolution, it will likely be more formidable. For now, I found the image quality lacking (reminded me of the stills mode found in camcorders five or six years ago--not so good) ...........
I've commented on this a number of times in the past, and whilst "interesting" is indeed a good word to describe it, don't expect major advances anytime soon.

As Charles has found out, the main issue which the makers omit from their hype is resolution. The other point is that the focus point is not infinitely variable - rather that it allows you to step backwards and forwards in finite steps. Fine - unless the point you want to hit focus on lies between two steps!! :-)

And the key point is that the resolution and the number of planes of focus multiply together to give the number of photosites on the base chip. If this is a fixed number, better refocussing ability can only come at the expense of resolution, and vice versa.

I'd expect the number of planes to be *at least* 16 - and if we even only go for 8 megapixel final resolution that means a photosite count of well over 120 megapixels! You may argue that it will come in time, as with other technology improvements, but there are fundamental limits, not least those due to the wavelength of light. That may be overcome by moving to much larger chip sizes - but don't expect it to be cheap!

The light-field concept is already being exploited by other companies for industrial etc applications where resolution is not the issue. (Where about 0.5-1.0 megapixel are adequate for the task.) Lytro are trying to expand it to the consumer world and "normal" photography - frankly, what they are marketing just isn't up to it, and more and more people are now finding that out. Improving the resolution to meet user expctation will be a far, far more difficult task than it ever was with normal digital cameras.

Who would need more resolution? Take a look at this article on what they can now do with still images. Lytro isn't just a refocusable stills image gimmick any more.

Lytro changed photography. Now can it get anyone to care? | The Verge (http://www.theverge.com/2014/4/22/5625264/lytro-changed-photography-meet-the-new-illum-camera)

From the article:

"If you look at a big-budget Hollywood production today, they’ll spend between 9 and 14 million dollars on just incremental hardware to shoot 3D, because you need multiple rigs. We can do all that in single-lens, single-sensor — that’s a big deal," Rosenthal says. "You look at the credits at the end of a movie and you see Camera Assistant 1, Camera Assistant 2, Camera Assistant 3… they’re doing focus pulls on set. If you can make that an after-the-fact decision, that’s a pretty big deal."

Of course to achieve that in practice and not just theory, Lytro would need to make a camera that records video. But that’s on the roadmap: "That’s something that largely gets solved as computational power continues on its Moore’s law rate of increase." Processors double in speed every two years, Moore says; Lytro’s perfectly positioned to take advantage of every increase.

Andrew

Who would need more resolution?

Well, I think resolution goes hand-in-hand with developments such as this.

Imagine in the future, you strategically set your cameras (possibly a selection of 4) on your set, all shooting very wide angles. They utilize this technology that Lytro is developing. On top of that, they are shooting 16K with an eye towards 4K distribution.

You would essentially just set the scene in motion and capture these 4 angles that cover pretty much everything given how wide they have been set to record. Then you take it to post, crop out to 4K the composition that you want, set your focus marks, and then place your lights (or really those three can be in any order you choose). Decide you need a dolly shot? You can add it. Want to change the way that shot rack-focused a mere 24 hours before release? Done.

This is where I see it ultimately going.

Video autofocus technologies have been available for decades, but professional sets still employ manual lenses and focus pullers. We already don't even need cameras, sets, or face actors, we can just do entire films as CGI in post. They don't happen very quickly, though.

While light field cameras will surely someday push into video, there's still a substantial sector of video creation that needs the right image direct out of camera and straight to the broadcast switchers or same-day edit desk, places where nobody's going to want to futz with the focal point.

Who would need more resolution? Take a look at this article on what they can now do with still images. Lytro isn't just a refocusable stills image gimmick any more.

But beware of what they DON'T say. The basic principle behind Lytro is the ability to trade resolution for refocussability etc, and yes, it may do what it says - to a point.

It does it - but as well as the much talked about resolution drop (final image compared to basic sensor) it's not infinitely refocussable as a basic lens would be. Think of it moving in steps, rather than a continuous smooth adjustment. You may be able to tap areas on sample images and get it to refocus between them - but you may not be able to focus between two planes, for example.

How bad is that? Well, the more you trade off resolution the smoother the post-focussing will be - one improves at the expense of the other. The only way to improve one without the trade off is to increase the number of individual sensors on the chip. In this respect it's a bit like taking a photo of the audience from the stage. You may want to set the point of focus on row 12 (which you could with a normal camera) - but with Lytro you may find you can post set it to row 9.... or row 15! But not precisely on row 12. And because the basic image resolution is relatively low (about 1 megapixel?) then in a way it helps, because you are less able to see such effects. But try to use it as a post tool on feature film quality material and you'll need a HUGE number of basic sensor elements. Even if possible, (and see next paragraph) the data storage requirements would be mindblowing - if you think 4k is bad......

Their CEO talks about Moore's Law in the quote - but unfortunately, that doesn't really hold up here. There is nothing he can do about the wavelength of visible light, and that limits the size of photosites on any chip, and hence the maximum possible number on any given size. The problem is one of the laws of physics - not technology or engineering. And we all know what Scottie had to say about that....... :-)

The article also talks about 3D, and whilst it's true light field technology gives some ability to "look around" an object, then again it's limited. I understand that the best you'll be able to do is imagine a cut out hole the diameter of the lens, and the effect will be as if you were looking through the hole and moved your eye around top-bottom, left-right. The perspective will change - but by a limited amount. To expect light field technology to revolutionise film production, being the technology that replaces current 3D hardware and on set focus pulling simply will not happen.

And light-field video cameras DO exist already - even if not via Lytro. See http://www.raytrix.de/tl_files/downloads/Raytrix_Slides.pdf for examples of what they are currently being actively used for. But their real use is coming about in industrial applications - not photography in it's more artistic sense. Think say of face recognition. You may not need huge resolution - but for such an application, one megapixel images with a depth map of the face may be vastly better than conventional 2D images at higher basic resolution. Think of medical uses.

Awesome post, especially with the info on the stepping limits for refocusing an image.

I had noticed that you never saw them specifying what size the final Lytro images actually were. It had to be bad if they were hiding it that much. For a moment I was thinking they might be able to make use of a 4K sensor chip. :-)

Andrew

Just to clarify, what I said above somewhat oversimplifies the situation to try to get the issues across, but the basic truth is that for a basic sensor with a given number of individual photosites, what light field technology does is sacrifice resolution for depth information. The more accurate you need the depth information, the more basic resolution you have to sacrifice. You have to rob Peter to pay Paul.

To keep even HD (say 2 megapixels) final resolution, with sufficient depth information for proper post focussing, you're likely to need at least 100 million photosites, which means a likely necessity for chips bigger than s35, and just think of the data rate if it's to read that at even 24 times a second!

But the real point is that there already are light-field (albeit not from Lytro!) video cameras. It's just that their resolution and performance are nowhere near good enough for "film-making". That's fine - they are aimed at industrial use and do that very well. But scaling the technology up to a level for film-making would be a huge challenge, if not impossibility - and as said before, it's more for "Laws of Physics" reasons than technology. That's the real problem.

Lytro's CEO throws in the towel, sadly:

"The goal now is to reshape VR, not consumer cameras."

The Lytro camera ceases to be...

Lytro CEO admits competing with Canon, Nikon, and smartphones was a losing game | The Verge (http://www.theverge.com/2016/4/4/11363186/lytro-ceo-no-more-consumer-cameras)

and

https://backchannel.com/war-stories-why-i-lit-up-lytro-b46124da32a6#.ugm6upj8x

And then steps back in the ring with a truly professional camera:

Lytro Cinema on Vimeo

So it seems like they've just dropped the consumer market, which was always a questionable one for the technology, and decided to focus on the professional market where they can potentially achieve what the technology has always promised.

More info on the camera here:

http://techcrunch.com/2016/04/11/lytro-cinema-is-giving-filmmakers-400-gigabytes-per-second-of-creative-freedom/

And they're even talking about resolution received from the camera! My, how things have suddenly changed.

Andrew

And they're even talking about resolution received from the camera! My, how things have suddenly changed.

Not really. All they really refer to is a basic "755 megapixel sensor", and from that someone has inferred "40K" as resolution. Do the maths and if you assumed 16:9 aspect ratio, it gives a horizontal photosite count of about 36.6K horizontally - which is where I assume the "40K" comes from.

But the whole point of the Lytro technology is that it trades resolution for other features (such as refocusing). So the final image resolution will be much less than 36K, the amount depending on the fineness of the "refocussability" etc. But it should certainly be up to at least 2K for video, probably even 4K, yet still have enough in hand to provide reasonably seamless post focussing. So far so good.

But ..... look at the figures again. I don't see any mention of sensor size? We're talking about the horizontal count going up roughly by a factor of about 9 compared to a standard s35 4K sensor, and it being read at 300fps. Either the photosite size must therefore decrease a lot, or the sensor size must go up. To keep the same photosite size, you're looking at a sensor of around 21x12 cms (wow! :-) ), and even if each was shrunk to only a quarter of the s35 4k size (so 1/16 the area), it's still about 5cmsx3cms in total. And if you're reading at 300fps, small photosites aren't a good idea.

Yes, it's possible, but even before thinking about the huge data requirements, expect it to be very, very expensive, and don't expect a zoom lens.

Certainly, don't expect it to be the future of general cinematography. (Not unless refocusability is worth more to you than zoom lens etc, before even thinking about the cost.)

The real question then is less will it work (I'm sure it will) - but will the additional functionality be worth the cost? (The article describes "This camera is essentially military-grade technology") Will it be simply more cost efficient, even for VFX work, to just use current techniques? Not as elegant maybe, but overall far more cost effective?

It's worth noting that they claim:
Lytro Cinema captures all the rays of light within a scene, providing a rich amount of Light Field data.
Well, not really. Such a camera is only capturing the rays that impact the camera lens, albeit the area at full aperture. And that becomes significant when you consider the 3D aspects Lytro refer to. It's quite true that such a light field camera can produce a more "true" 3D effect than a stereoscopic camera - but the limits of such are similar to looking at a real world scene through such as a hole in a fence. Perspectives will change somewhat depending which part of the hole you look through - but the limits will be set by the edges of the hole. In this case, the diameter of the lens corresponds to the hole, and limits the amount of "looking around" objects that's possible.