Lossless MJPEG exports?

[Pierre Aguillard]

We've just finished shooting a feature in MJPEG at the resolution of 4096 x 2160 with a Canon 1DC and are about to start cutting with FCPX on a Retina Macbook Pro with 16GB of RAM - so far viewing the rushes and making preliminary cuts has posed no problems and we're particularly pleased there is no lag or slowing down of the footage which simply plays like it was any other smaller format…

I have a question about MJPEG and FCPX and my own cutting methodology. When cutting, I like to copy quite a bit, so to speak. For instance, I will focus on a particular scene, cut it together and render it. Then will take the rendered file and design its sound, lay it all down, then render that. I will do this, individually for each scene in the film. Then I will dump the fifty or so scenes into a fresh timeline and render that also. Subsequently, I will lay in the score and re-render. So, as you can see, there are many generations of copying / rendering / exporting.

In the old days, I would have been going back to the neg or in analog video using a timecode and go back to the masters… but I prefer not to go through a similar process here. Therefore, what I need to know is whether the duplication I have outlined above will cause any degradation to my image whatsoever. To be totally clear, it's all MJPEG files at 4096 x 2160 and every single time I export something I am religiously doing it with all the best, highest, slowest settings and outputting at 4444 and so on - with so much reverence that it's almost as if I am handling a negative.

But is it possible I will lose anything from the image, even the slightest bit of quality, doing what I have described? If so, I will have to find another way. I would really appreciate any advice (or reassurance?) you experienced FCPX folks can throw my way, thank you for it deeply in advance and ask you to please remember this is not a request for help with cutting methodologies and so on (I have that part figured out over the years) but rather a straightforward technical query about whether repeatedly exporting MJPEG will cause the image to slowly, even slightly degrade, thus making my methodology a faulty one!

Is there, for instance, a particular setting or settings that will ensure the MJPEG remains as MJPEG until such time as I want to create various versions of the master later on?

I need to be darn sure that my master is 100% for many obvious reasons, not least of all that we'll be premiering in 4K cinemas.

[John Nantz]

Hi Pierre - You posted a very good question and what is appreciated is all the details. I've been watching the post to see what one of the local gurus would reply but I'm disappointed in the lack of interest. However, I'll take a stab at a reply.

First off, though, an important Disclaimer. Although I've been around computers and even started as a programmer back in the days of punched cards and when a 4K (yes, that's a K) machine took up a large room, I sure don't claim to be a guru. Another disclaimer item is that I'm just a hobbyest and not a pro. Okay, so that is out of the way.

As I see it, we're probably talking about all the "0s" (zeros) and "1s" (ones), and whether they're all there in the place they're supposed to be.

There is such a thing as ECC (error correcting code) memory and some computers have it and some don't. Typically ECC memory is found on servers so that tells me another thing. If there wasn't an issue with the 0s and 1s then ECC wouldn't be needed so there must be some kind of issue there.

Hard drives are basically a collection of very little tiny magnets with the 0s on one end and the 1s on the other end (kinda like north & south).

So, what if one of these little magnets didn't flip like it was supposed to? If we're talking abut a picture maybe one of the little pixels wouldn't be what it is supposed to be. I'm just guessing.

Another thing that comes up in rendering is the area at the end of a section (I don't know what they call it). Anyway, that edge area can be rendered differently depending on the application and I suppose that maybe even with subsequent re-rendering. Again, just another guess.

My workflow is to keep rendering to a minimum and that means that background rendering is turned off. The other thing I try to do is minimize is the amount of editing - trying to get it "right" the first time. And this is even though I'm often accused of being a perfectionist.

Artifacts: Do these misplaced 0s and 1s cause artifacts in the final product? That's a good question. They are definitely known to cause artifacts when burning to a DVD but I don't know if editing or the burning process on the medium, or both, causes them.

Maybe this will stir up some discussion.

[William Hohauser]

Having never worked in MJPEG on this level, I can't claim to have any working experience. I was waiting to see if anybody had something to relate but since this hasn't occured and the question is very esoteric let me throw a bit of working knowledge of FCPX into the discussion.

If you are indeed working MJPEG uncompressed (and you seem to have experience in this), then yes, you should be confident that your renders should be free of any generational artifacts. However, MJPEG is a compression algorithm designed to save space as opposed to something like a Photoshop file and I have no idea if it can be set to never compress an image. Years ago when I would work in AfterEffects the usual plan was to work the files in the Animation codec (which are huge) which would guarantee us a clean final result. How many generations can MJPEG at it's highest setting take before introducing a compression artifact is beyond my experience. Whether anybody can see that artifact is another question. Certainly years ago I experienced MJPEG artifacts but I was working BetaSP grade SD where hard drive space was limited and very expensive. These days I usually work in ProRes and never experience a visible compression problem, even when projected in a theater. Perhaps I am not looking close enough but then neither are my clients.

FCPX has been designed around the ProRes codec model which is an industry standard now. Finished independent films are sometimes distributed in ProResHQ which projects perfectly after conversion to DCP (which is in JPEG2000, another compression codec). ProRes4444 was designed for effects and exacting image manipulation. If you are sharing scenes with other editors outside of FCPX then rendering the files makes sense and exploring the highest quality makes sense. However there are functions within FCPX that you can experiment with to see if it works with your editing style and the project. Compound clips are a great way to isolate sequences within the project and this might alleviate the need to render sequences.

[Craig Seeman]

I'm not quite sure I completely understand the reasoning in your workflow but I do have a background in compression. ProRes4444 is a compressed (lossy) codec.

I haven't worked with MJPEG in FCPX but if you have MJPEG .mov it may work.

If you're rendering and exporting you are going through generational loss. It may be slight but it's the nature of lossy codecs.

If you really don't want generational loss, one might edit a proxy file and link back to the camera masters for a final render.

[Pierre Aguillard]

Thanks to everybody for taking the time to reply, I've been chatting to some other practitioners about this and have learnt a few things...

Firstly, I've discovered how robust the Pro Res 4444 is supposed to be. It seems that if I do what is described in my initial post, each time rendering to Pro Res 4444 with all the settings at the slowest and best, it's really unlikely any degradation will be visible to the human eye. From what I understand, after the first render, I will be in a Pro Res 4444 context and further degradation is not really, effectively, going to occur at least in terms of human visual perception. Some say you can copy the material 10 times without noticing any difference, and Apple themselves seem to imply more, almost as if quality-loss just ceases because you're using Pro Res 4444. I would like to know if this sounds right to you all, or whether it's marketing/fanboy stuff?

Secondly, I've learnt about an FCPX plug in called, ironically, 'Compressor', which allows for straight file-copying type results, in which the 1s and 0s of each frame are preserved perfectly, and only the actual order of the frames is changed, depending on your edit choices - theoretically this should be possible with MJPEG because the compression is intra frame compression (compression within each image) and not inter frame compression (compression that works between images, often meaning individual frames are 'missing' or guessed etc.) but to be honest I would love to know if anyone has actually used Compressor to create lossless or even just used Compressor at all...

Finally, I've been told rather forcefully that rendering and exporting is always lossless and there is no such thing as lossless NLE cutting, particularly when using a 'lossy' format like MJPEG - the person saying this took the position that it doesn't matter in the slightest that MJPEG is intraframe, and claimed it's impossible to simply re-arrange the frames intact, because there will always be recompression...which doesn't tally with my experience and is something else I would like your thoughts on - particularly yours Craig!

Deeply appreciated everyone... :)

[David Knaggs]

Hi Pierre.

Compressor is a must, in my opinion. It's not a plug-in to FCP though. It's a standalone application, made by Apple and you could use it to compress QuickTime movies into various formats without ever touching FCP. But FCP itself is set up to export from your Timeline directly to Compressor (provided you've purchased Compressor, of course).

Compressor has 2 uncompressed export settings to choose from: Uncompressed 10-bit 4:2:2 or Uncompressed 8-bit 4:2:2 (both with audio pass-through).

Good luck with your project.

[William Hohauser]

If you leave the footage alone in FCPX then any exported file will just be a simple reordering of the frames, no re-rendering should occur but as soon as you change the smallest thing, it will be re-rendered. You do not need Compressor to achieve this, in fact you do not need Compressor at all if your intention is to edit just a feature. You might need a plug-in to export the audio for an audio editor but everything else is within FCPX at this point. Color correction might need to be done outside the program so you will have to explore your methods to deliver the original footage or the edited project to the colorist.

ProRes is intra-frame just like MJPEG and does not have some of the compression quirks that MJPEG can exhibit although at 4K I'm sure those are minimal.

[Pierre Aguillard]

Thanks for your responses, William.

When you write "if you leave the footage alone in FCPX then any exported file will just be a simple reordering of the frames" it really engages with the spirit of what I am trying to say - and actually I may not have made this clear enough in my initial post...

I don't really mess with the image when cutting. I generally favour straight cuts and don't go in for effects. Indeed, grading won't be a major process for us either, as we favoured Canon Portrait over Canon Log and got the results we wanted in camera. Obviously we'll grade, but only where problems occur.

So I am hoping that by rendering to Pro Res 4444 I will 'catch' the quality pretty much at its original level and not degrade it thereafter, which is to say not encounter degradation at all in this process really and it seems the answer to my original question may be something like: technically, there is loss but in reality there is not!

[William Hohauser]

If you have a pro screening facility willing to help out, think about working on the film in ProRes4444 for a few scenes and bring the tests to watch projected compared to the same footage in MJPEG. Or do the same test on a quality calibrated monitor not a standard computer monitor. You'll learn a lot from that.

[Pierre Aguillard]

Yeah, I've been thinking about that too - after all I'm hardly going to be able to judge this in 1080p!

Feeling like I have reached my comfort level, though and that MJPEG to ProRes 4444 will be stable enough for me.