Light Field Technology!

[Nevin Styre]

I'm not talking about recording video. I mean animating still pictures in after effects for effect in a video production. For title/full-screen supers, diagram purposes, etc.

Say for a concert promo video/commercial. I can see using concert pictures taken with lytro, animating these still photos by rack focusing through thousands of people stopped in 1 moment in time.

[Alister Chapman]

You could do that now by taking an image with a very deep DoF and then selectively de-focussing the image to simulate out of focus or shallow DoF.

Lytro is still an interesting technology, but the flaw is that the camera is still using a lens. If there is a lens, then it's bringing light in to focus at certain points or over a certain range. I suspect the technology that will truly give a focus anywhere after capture ability won't use a lens as we know it.

[David Heath]

Originally Posted by Nevin Styre

I can see using concert pictures taken with lytro, animating these still photos by rack focusing through thousands of people stopped in 1 moment in time.

Sorry for misunderstanding - I can see what you mean, I suppose a variation on the theme of tracking round a "frozen in time object".

It's an interesting thought, but I'm still not sure the lytro technology is even up to that, not for quality work anyway. You'd have to check it in practice, but I suspect it won't be quite high enough resolution, and give a steppiness to the refocussing. So with (say) 100 rows of seats the effect would be more like being able to have focus on every fifth row and mixing between these planes of focus - not a smooth rack. (Imagine having 16 cameras all focussed at different planes - that's equivalent to the raw material you have to work with.) I would be interested to see it in practice.

[Mark Watson]

I'd think they could incorporate this ability into dSLRs in a manner akin to exposure bracketing. Would be nice to tweak focus after the fact when doing a timelapse. Canon???

Mark

[Chris Soucy]

Lytro Camera Produces Amazing, Interactive Pictures (PHOTOS)

Interesting.


CS

[Charles Papert]

Sort of, I guess.

I had one on pre-order, got it this spring. Tried it out for a day or two and sent it back.

It's an interesting concept and as the capabilities progress (3D) along with the resolution, it will likely be more formidable. For now, I found the image quality lacking (reminded me of the stills mode found in camcorders five or six years ago--not so good) and the ergonomics funky (novel and attractive package, but the touch sensitive zoom control was erratic).

I'll keep an eye on them but I can't recommend the first gen product; it's kind of a one trick pony. You go walking around searching for things to shoot on extreme planes to take advantage of the depth of field thing, which isn't really photography, more like a novelty.

My opinion of course.

[David Heath]

Originally Posted by Charles Papert

It's an interesting concept and as the capabilities progress (3D) along with the resolution, it will likely be more formidable. For now, I found the image quality lacking (reminded me of the stills mode found in camcorders five or six years ago--not so good) ...........

I've commented on this a number of times in the past, and whilst "interesting" is indeed a good word to describe it, don't expect major advances anytime soon.

As Charles has found out, the main issue which the makers omit from their hype is resolution. The other point is that the focus point is not infinitely variable - rather that it allows you to step backwards and forwards in finite steps. Fine - unless the point you want to hit focus on lies between two steps!! :-)

And the key point is that the resolution and the number of planes of focus multiply together to give the number of photosites on the base chip. If this is a fixed number, better refocussing ability can only come at the expense of resolution, and vice versa.

I'd expect the number of planes to be *at least* 16 - and if we even only go for 8 megapixel final resolution that means a photosite count of well over 120 megapixels! You may argue that it will come in time, as with other technology improvements, but there are fundamental limits, not least those due to the wavelength of light. That may be overcome by moving to much larger chip sizes - but don't expect it to be cheap!

The light-field concept is already being exploited by other companies for industrial etc applications where resolution is not the issue. (Where about 0.5-1.0 megapixel are adequate for the task.) Lytro are trying to expand it to the consumer world and "normal" photography - frankly, what they are marketing just isn't up to it, and more and more people are now finding that out. Improving the resolution to meet user expctation will be a far, far more difficult task than it ever was with normal digital cameras.

[Andrew Smith]

Who would need more resolution? Take a look at this article on what they can now do with still images. Lytro isn't just a refocusable stills image gimmick any more.

Lytro changed photography. Now can it get anyone to care? | The Verge

From the article:

"If you look at a big-budget Hollywood production today, they’ll spend between 9 and 14 million dollars on just incremental hardware to shoot 3D, because you need multiple rigs. We can do all that in single-lens, single-sensor — that’s a big deal," Rosenthal says. "You look at the credits at the end of a movie and you see Camera Assistant 1, Camera Assistant 2, Camera Assistant 3… they’re doing focus pulls on set. If you can make that an after-the-fact decision, that’s a pretty big deal."

Of course to achieve that in practice and not just theory, Lytro would need to make a camera that records video. But that’s on the roadmap: "That’s something that largely gets solved as computational power continues on its Moore’s law rate of increase." Processors double in speed every two years, Moore says; Lytro’s perfectly positioned to take advantage of every increase.

Andrew

[Gary Huff]

Originally Posted by Andrew Smith

Who would need more resolution?

Well, I think resolution goes hand-in-hand with developments such as this.

Imagine in the future, you strategically set your cameras (possibly a selection of 4) on your set, all shooting very wide angles. They utilize this technology that Lytro is developing. On top of that, they are shooting 16K with an eye towards 4K distribution.

You would essentially just set the scene in motion and capture these 4 angles that cover pretty much everything given how wide they have been set to record. Then you take it to post, crop out to 4K the composition that you want, set your focus marks, and then place your lights (or really those three can be in any order you choose). Decide you need a dolly shot? You can add it. Want to change the way that shot rack-focused a mere 24 hours before release? Done.

This is where I see it ultimately going.

[Kevin McRoberts]

Video autofocus technologies have been available for decades, but professional sets still employ manual lenses and focus pullers. We already don't even need cameras, sets, or face actors, we can just do entire films as CGI in post. They don't happen very quickly, though.

While light field cameras will surely someday push into video, there's still a substantial sector of video creation that needs the right image direct out of camera and straight to the broadcast switchers or same-day edit desk, places where nobody's going to want to futz with the focal point.

[David Heath]

Originally Posted by Andrew Smith

Who would need more resolution? Take a look at this article on what they can now do with still images. Lytro isn't just a refocusable stills image gimmick any more.

But beware of what they DON'T say. The basic principle behind Lytro is the ability to trade resolution for refocussability etc, and yes, it may do what it says - to a point.

It does it - but as well as the much talked about resolution drop (final image compared to basic sensor) it's not infinitely refocussable as a basic lens would be. Think of it moving in steps, rather than a continuous smooth adjustment. You may be able to tap areas on sample images and get it to refocus between them - but you may not be able to focus between two planes, for example.

How bad is that? Well, the more you trade off resolution the smoother the post-focussing will be - one improves at the expense of the other. The only way to improve one without the trade off is to increase the number of individual sensors on the chip. In this respect it's a bit like taking a photo of the audience from the stage. You may want to set the point of focus on row 12 (which you could with a normal camera) - but with Lytro you may find you can post set it to row 9.... or row 15! But not precisely on row 12. And because the basic image resolution is relatively low (about 1 megapixel?) then in a way it helps, because you are less able to see such effects. But try to use it as a post tool on feature film quality material and you'll need a HUGE number of basic sensor elements. Even if possible, (and see next paragraph) the data storage requirements would be mindblowing - if you think 4k is bad......

Their CEO talks about Moore's Law in the quote - but unfortunately, that doesn't really hold up here. There is nothing he can do about the wavelength of visible light, and that limits the size of photosites on any chip, and hence the maximum possible number on any given size. The problem is one of the laws of physics - not technology or engineering. And we all know what Scottie had to say about that....... :-)

The article also talks about 3D, and whilst it's true light field technology gives some ability to "look around" an object, then again it's limited. I understand that the best you'll be able to do is imagine a cut out hole the diameter of the lens, and the effect will be as if you were looking through the hole and moved your eye around top-bottom, left-right. The perspective will change - but by a limited amount. To expect light field technology to revolutionise film production, being the technology that replaces current 3D hardware and on set focus pulling simply will not happen.

And light-field video cameras DO exist already - even if not via Lytro. See http://www.raytrix.de/tl_files/downl...rix_Slides.pdf for examples of what they are currently being actively used for. But their real use is coming about in industrial applications - not photography in it's more artistic sense. Think say of face recognition. You may not need huge resolution - but for such an application, one megapixel images with a depth map of the face may be vastly better than conventional 2D images at higher basic resolution. Think of medical uses.

[Andrew Smith]

Awesome post, especially with the info on the stepping limits for refocusing an image.

I had noticed that you never saw them specifying what size the final Lytro images actually were. It had to be bad if they were hiding it that much. For a moment I was thinking they might be able to make use of a 4K sensor chip. :-)

Andrew

[David Heath]

Just to clarify, what I said above somewhat oversimplifies the situation to try to get the issues across, but the basic truth is that for a basic sensor with a given number of individual photosites, what light field technology does is sacrifice resolution for depth information. The more accurate you need the depth information, the more basic resolution you have to sacrifice. You have to rob Peter to pay Paul.

To keep even HD (say 2 megapixels) final resolution, with sufficient depth information for proper post focussing, you're likely to need at least 100 million photosites, which means a likely necessity for chips bigger than s35, and just think of the data rate if it's to read that at even 24 times a second!

But the real point is that there already are light-field (albeit not from Lytro!) video cameras. It's just that their resolution and performance are nowhere near good enough for "film-making". That's fine - they are aimed at industrial use and do that very well. But scaling the technology up to a level for film-making would be a huge challenge, if not impossibility - and as said before, it's more for "Laws of Physics" reasons than technology. That's the real problem.

[Chris Hurd]

Lytro's CEO throws in the towel, sadly:

"The goal now is to reshape VR, not consumer cameras."

The Lytro camera ceases to be...

Lytro CEO admits competing with Canon, Nikon, and smartphones was a losing game | The Verge

and

https://backchannel.com/war-stories-...2a6#.ugm6upj8x

[Evan Donn]

And then steps back in the ring with a truly professional camera:

Lytro Cinema on Vimeo

So it seems like they've just dropped the consumer market, which was always a questionable one for the technology, and decided to focus on the professional market where they can potentially achieve what the technology has always promised.

More info on the camera here:

http://techcrunch.com/2016/04/11/lyt...ative-freedom/