Best HDV workflow.

[Jeff Turkali]

Originally Posted by M. Paul El-Darwish

Following up on Jason's lucid comparison, the questions that an editor might ask themselves are:

- How noticeable to the customer is the degradation of quality in areas of the edit that have been filtered/transitioned? I guess three clips from the three different options would be great to review.
- Is the additional disk space that's required for a totally ProRes edit paradigm always worth the extra expense or extra physical drive arrays?
- Seems like the pure HDV and mixed capture/edit paradigm advantage laptop-based edit turnkeys?

It depends on the customer, but I found out that the degradation is heartbreakingly bad. Like the difference of going from HDV to SD. You are knocked down a flight of stairs in quality.

Yes, it is worth the cost of a cheap drive to capture into Prores and work that way.

I did only simple wipes and desolves, a few fade ins and fade outs, and it ruined my HDV footage. I will do that entire edit over again.

[Robert Bec]

Originally Posted by Jeff Turkali

Incorrect, the capture from FW is the HDV signal that get printed to Prores format.

It is like transfering one format to another format, you are not capturing Prores, but transfering to it while capturing.

Ok So i will capture Prores HQ which i already tried through my canon xh-a1 data rate was around 18mb/sec and capturing in HDV data rate was at 3mb/sec

So i guess capture in Proress ,edit in Prores and export i guess in what ever format i need or in pro res

[Jeff Turkali]

Originally Posted by Robert Bec

Ok So i will capture Prores HQ which i already tried through my canon xh-a1 data rate was around 18mb/sec and capturing in HDV data rate was at 3mb/sec

So i guess capture in Proress ,edit in Prores and export i guess in what ever format i need or in pro res

Yes, this is the way to go like a "Pro". Some folks (posts in this forum) will export to Prores, and then take that finished file and drop it into a SD timeline to create a new SD file. That is just one way to create a SD file out of many.

But bottom line is that capturing it originally in Prores is getting you off to the correct start. Drives are cheap, so no reason to delay your workflow.

[Luke Oliver]

yes i also think pro res is the way to go

i have spent the last 5 months trying and testing every combination and the best results are, or in my opinion, capture pro res 422, edit pro res 422 ,1480 x1280 25p, export apple prores 1480x1280 25p, drop than into compressor,tweak up the dials, best quality SD DVD 90 mins , then into dvd studio for output.

The quality is fantastic. Almost looks HD on my 42inch, so sharp.

Also isnt pro res better with color, im sure i read that but.........


luke

just need some terabyte drives now

[David McGiffert]

Jason,

That was a great reply.
It is the workflow I have been using for a year and a half
and it works very well.

David

[William Hohauser]

Originally Posted by Robert Bec

Dont you need a card like decklink or kona to capture prores it cant be done via firewire is this correct?

Depends on the model of Mac you have. The Dual 2g G5 I have can capture 720p30 to ProRes directly thru FirwWire but not 1080i. Final Cut seems to able to assess your computer and will give you the option.

[William Hohauser]

Originally Posted by Jeff Turkali

I just read your post Jason, as this forum was not sending emails when posts are followed up on.

But you in your wonderful explaination of pros and cons of Prores/HDV codecs did not address the issue I brought up. We are talking about editing, and not what are the shortcomings in HDV at the camera stage.

And that is that rendering out to Prores after editing/composing/transitions, etc. defeats the purpose of Prores. My posts stated that you must get in Prores before doing these things to take advantage of the codec, and not simply render out to prores.

You did not address this issue. But nobody seemed to notice anyway.

Your understanding of the process needs a little sorting out. The shortcomings of HDV are after the camera, it's in the recording process and the editing process.

When you edit in HDV you are cutting the unadulterated video files direct from tape or camera. That is an m2t file (GOP-based MPEG2 transport stream) that has a QuickTime wrapper placed around it to enable frame by frame editing. The m2t file is untouched at this point if you capture it via FCP or use ClipWrap on the raw m2t file. No quality is lost or gained here. A cuts only edit in HDV is as good as HDV will ever get. Once you apply a transition or filter, the m2t file has to be re-encoded back to a m2t file. As we all know, this doesn't lend itself to clean renders and the process requires a lot of system resources.

ProRes is a frame-based codec not a GOP codec like HDV so it requires less system resources even though the file sizes are larger and it works in a larger color space. By transcoding HDV into ProRes you get all the benefits of a frame based coded and the fact that ProRes employs less compression than HDV therefore all renders will come out cleaner. But....

Transcoding is not a transparent process. You will loose a very minor amount of quality but the new files keep the resulting quality high during rendering. Also you are rendering in a better color space than in HDV. That helps a lot.

By setting your timeline to HDV with ProRes renders you keep the quality of your cuts only footage and the renders are high quality without using tons of disk space. FCP plays the footage seamlessly. This can be outputted to HD tape via a number of methods or rendered via Compressor for DVD or to Uncompressed files.

This process does not defeat the purpose of ProRes, it takes advantage of the very qualities that make ProRes an attractive codec. If you are working with specific input/output hardware that uses ProRes, then stick with a full ProRes work flow otherwise you are wasting drive space in a situation that you have described as drive space sensitive.

Try a test edit with this method. If it doesn't work for you then it doesn't. It works for a lot of other people.

[Jason Livingston]

I'll expand a little on my original post and what William (correctly) stated.

For the quality comparison numbers, let's say 10 is the max (pure uncompressed quality), each pass of HDV reduces the quality by 4, and each pass of ProRes reduces the quality by 1. (These have no connection to any real-world metric and are for demonstrative purposes only).

--- Pure HDV ---
In the pure HDV workflow, the files captured by firewire have 0 quality loss (compared to the compressed HDV stream recorded by the camera), it is just like making a copy of a file. When you play back HDV on your timeline, you still have 0 quality loss. When you need to render a change, this is what happens. First each frame is decompressed to an uncompressed frame in memory (neither losing nor gaining any quality). The filter/title/etc. functions are performed, and then the modified frame is re-compressed into the HDV codec (causing a large loss in quality due to the limitations of the HDV codec). But frames which are unchanged don't go through this process. They retain exactly the same quality that the camera recorded with 0 loss.

Pure HDV Summary:
Unchanged parts: HDV artifacts (Quality: 6)
Changed parts: HDV artifacts + more HDV artifacts (Quality: 2)


--- Pure ProRes ---
In the pure ProRes workflow, as you are capturing the video over firewire, each HDV frame is decompressed into an uncompressed frame buffer (neither losing nor gaining quality), and then compressed into the ProRes codec (causing a slight quality loss, because ProRes is a lossy codec). That is to say, every single frame of your video captured in this way is going to lose some quality. Not a huge amount, but it can mathematically be proven that some data WILL be lost. This is the nature of any lossy codec, which is what ProRes is. When you apply effects/filters, each frame is decompressed from ProRes, has the effects applied, and then recompressed into ProRes (causing a further slight loss).

Pure ProRes Summary:
Unchanged parts: HDV artifacts + ProRes artifacts (Quality: 5)
Changed parts: HDV artifacts + ProRes artifacts + more ProRes artifacts (Quality: 4)


--- Mixed HDV/ProRes ---
In the mixed HDV/ProRes workflow, frames which are unchanged will retain their original data and lose no quality. Frames which need to be changed will be decompressed from HDV into an uncompressed buffer in memory, have the effects applied, and then re-compressed into ProRes (causing a slight loss, but much less than when compressing HDV).

Unchanged parts: HDV artifacts (Quality: 6)
Changed parts: HDV artifacts + ProRes artifacts (Quality: 5)


--- Bottom Line ---

The mixed HDV/ProRes workflow gives you the best possible quality (outside of resorting to uncompressed), with only a minor decrease in real-time editing speed and minor increase in disk usage.

The pure ProRes workflow gives you an increase in real-time editing speed, at the cost of slightly decreased quality and greatly increased disk usage.

The pure HDV workflow has the least disk usage but the worst quality (unless doing cuts-only editing) and the worst real-time performance.

The ONLY advantage to the pure ProRes workflow is if you absolutely need the maximum possible number of real-time effects that can be played without rendering. Any sections that need to be rendered will have a slight quality LOSS compared to the mixed HDV/ProRes workflow. Any sections that don't need to be changed will also have a slight quality LOSS compared to either the HDV or mixed HDV/ProRes workflows.

[Jeff Turkali]

Originally Posted by Jason Livingston

Jeff,
When working with HDV, the compression is done in the camera before the data is written to tape or outputted via firewire. The loss due to HDV has already occurred. When you transfer HDV to your NLE by firewire, there is no additional loss (just like copying a file bit-for-bit).

Of course there is loss that has already occurred as HDV is a mp2 compressed file. But there is about to be more loss during editing with any transitions, etc., unless you get it into another codec quickly.

I never said that there was loss by transferring the files bit by bit, I said nothing of the sort.

Originally Posted by William Hohauser

Your understanding of the process needs a little sorting out. The shortcomings of HDV are after the camera, it's in the recording process and the editing process.

The recording process is in the camera just as Jason says above. The compression happens there also.

With these long detailed posts, there still is not the suggestion that you must get into Prores right away to avoid the drop in picture quality. That was my statement, and it remains my workflow. I don't know any editors that do only cut, insert, cut, insert - either. All my projects require something more than just a simple cut.

Professionals who work with HDV always get into the codec that they plan in working in at the earliest stage of the project. That is my point and I am sticking to it. We can go around and around about what degrades the picture quality worse (cuts or transitions), where Prores' qualities are most apparent, and how well it works with color corrections, etc.

The bottom line is that you want to get into your working codec early as possible (which ever one it will be), this has been taught to me from working professionals, and I believe them. All this talk about where the compression (and loss) happens in HDV is a side issue concerning cameras and the way they record, and if it happens in or after camera is of lesser concern. Jason says in, William says after.

The beginners who are asking about Prores workflow need to know to get into it at capture if they are at all able to, and they plan on doing any serious editing. That is the important issue concerning workflow.

[William Hohauser]

Sorry, you are misunderstanding the process still.

A camera is a camera. It has it's qualities, good or bad. Back in the day of separate camera / VTR packages, you could take a great $50,000 camera (intended for BetaSp, 1 inch) and hook it up to a portable VHS and get the best VHS recording possible but it still had all the problems of VHS. These days a camera is usually wedded to a specific recording technology, DVCam, HDV, XDCam, whatever. A good 3-chip HDV camera might be making better images than the format can show but unless you are willing to go thru some difficult rigging, HDV is what this camera is going to shoot. Maybe there will be a ProRes recording camera but now there isn't.

Rendering issues have nothing, please take note, nothing to do with the camera. It's all about the recording codec.

Maintaining quality from HDV footage is what Jason and I are talking about. There are many work methods, you asked and we suggested. Please don't be offended if we did not concur with what some other professionals told you. They are not wrong and neither are we. Are you at least going to try the test?

In the end, if going straight to ProRes fits your editing style and works with your other editors, great. But it's not the only way to work and please do suggest that we are misleading beginners because we are not. The methods put forth by Jason and expounded upon by me are serious editing workflows, used by professionals. Us.

[Jeff Turkali]

Originally Posted by William Hohauser

Rendering issues have nothing, please take note, nothing to do with the camera. It's all about the recording codec.

Maintaining quality from HDV footage is what Jason and I are talking about.

Then why are you going on about cameras? And your tone is insulting.

[Michael Lafleur]

Originally Posted by William Hohauser

...Maintaining quality from HDV footage is what Jason and I are talking about. There are many work methods, you asked and we suggested. Please don't be offended if we did not concur with what some other professionals told you. They are not wrong and neither are we. Are you at least going to try the test?

In the end, if going straight to ProRes fits your editing style and works with your other editors, great. But it's not the only way to work and please do suggest that we are misleading beginners because we are not. The methods put forth by Jason and expounded upon by me are serious editing workflows, used by professionals. Us.

Jason & William,

Many thanks for shedding light on what, for me, is still a somewhat mysterious process of achieving best results (i.e. min loss of quality/res) while editing HDV. Jason, you ought to be a teacher with your gift of clarity!

I have one question for both of you and forgive my ignorance in advance. Here’s my scenario and then the question will follow: I shot a multicam wedding last year using three JVC HD100 cameras and captured my 720-30p footage into FCP2 as per standard methods. However, I completed most of the editing last month BEFORE I was enlightened by your recommended HDV workflow, adding my transitions and filters along with multi-camera overlays/masks/feathering, etc, while leaving the “Render control” sequence settings to “Same as Sequence Codec” (i.e. I never knew the advantages of switching them to ProRes before I started editing my project).

My question is, now that I have already added transitions, etc. CAN I “RECOVER” THE LOSSES IN QUALITY caused by allowing FCP to render in HDV if I go back and change the sequence render settings to ProRes and then re-render the project, and so on?

If not, (sorry, a second question), how might I go about re-doing my edits without starting from scratch? (i.e. do I simply remove all transitions and/or re-import files and footage onto my timeline and/or cut and paste everything into a new sequence set to render in ProRes, and/or…??)

Thanks so much in advance for any further illumination.

Michael Lafleur

[Jason Livingston]

Originally Posted by Jeff Turkali

Of course there is loss that has already occurred as HDV is a mp2 compressed file. But there is about to be more loss during editing with any transitions, etc., unless you get it into another codec quickly.

The way to minimize loss is to keep the unchanged HDV segments in HDV so that they have zero loss on the timeline compared to the original. Changing them into another codec, no matter how quickly, will only cause unnecessary loss (unless you're talking lossless or uncompressed codecs, which ProRes is not). For rendered/changed segments, you want to minimize the number of lossy codec cycles the video has to go through. Using an all-ProRes workflow adds an additional lossy cycle which in most cases is unnecessary.

Professionals who work with HDV always get into the codec that they plan in working in at the earliest stage of the project. That is my point and I am sticking to it.

I think it's unfair to imply that "[all] professionals" "always" use the workflow that you prefer. I happen to know of many professionals that would disagree. There certainly are some cases for which the all-ProRes workflow is best (if real-time performance or rendering time is the most important concern), but in doing so you are sacrificing quality and disk space.

The beginners who are asking about Prores workflow need to know to get into it at capture if they are at all able to, and they plan on doing any serious editing. That is the important issue concerning workflow.

I think that beginners should choose the workflow that works best for them and meets their needs given the pros and cons of each that I outlined in my post above, instead of blindly sticking to a single workflow based on mistaken or incomplete information.

[Jason Livingston]

Originally Posted by Michael Lafleur

My question is, now that I have already added transitions, etc. CAN I “RECOVER” THE LOSSES IN QUALITY caused by allowing FCP to render in HDV if I go back and change the sequence render settings to ProRes and then re-render the project, and so on?

Yes, you may need to trash your render files to force FCP to re-render, but that's all. Your original source media remains untouched in the capture files, so by re-rendering all of the effects and filters will be re-calculated from the pristine sources and you'll generate new, high-quality renders. Any un-changed parts of your timeline will stay in their pristine source format(s) and not suffer any generation or rendering loss at all.

[Michael Lafleur]

Originally Posted by Jason Livingston

Yes, you may need to trash your render files to force FCP to re-render, but that's all. Your original source media remains untouched in the capture files, so by re-rendering all of the effects and filters will be re-calculated from the pristine sources and you'll generate new, high-quality renders. Any un-changed parts of your timeline will stay in their pristine source format(s) and not suffer any generation or rendering loss at all.

Thank you for such a prompt and helpful reply! I have never tried to purge render files on a project but I assume they are easy to locate on my scratch disk and the process is quite straightforward. If I need some assistance, I will ask for some basic coaching once again.

Kudos to You from the Frozen North