Anyone encoded HDV to H.264 streaming?

[Kaku Ito]

Has anyone encoded HDV to QuickTime 7 H.264 streaming like how well Apple has done? I have tried to use the preset in the Compressor 2, but the results in dropping frames often when you playback through ADSL connection or FTTH connection.

Does anyone know Apple's trick?

[Jack Zhang]

The trick is a progressive download or (Fast Start) header that allows the player to stream the MOV file up to the point of downloading. All you need to do is upload the file, and after the entire file downloads, you can play the file with no dropouts. Try playing one of the HD gallery videos (especially the NASA video) and see for yourself how it progressively downloads.

[Kaku Ito]

Jack,

It is not the fast start download file I'm talking about, I'm talking about the keynote speech that they streamed for the WWDC. It is streaming, not fast download, but it looks very good. Thank you anyway.

[Allen Lu]

Doesnt the streaming require the Quicktime server?

HD streaming works with Windows Media Server and negotiates an optimum bandwidth to stream.

[Jeff Sayre]

Kaku:

I'm having the same problem. I have just set up QTSS on my dedicated server and later this afternoon will try streaming a H.264 version of a movie short I produced.

If you are using QTSS, you need to make sure that your file is hinted for streaming, not set for Quick Start.

However, I think that there is something else preventing us from getting the quality we want. I've being messing around with various compression options in Compressor 2.0.1 for about three days now, trying to figure out the optimum settings. HDV is just a different format to conquer. Obviously it's possible, but I cannot yet figure it out.

[Murad Toor]

Originally Posted by Kaku Ito

Jack,

It is not the fast start download file I'm talking about, I'm talking about the keynote speech that they streamed for the WWDC. It is streaming, not fast download, but it looks very good. Thank you anyway.

Part of their trick was to half the framerate, if I recall correctly.

[Kaku Ito]

Murad,

I will try to do that and provide it on my QTSS, so you folks can see.

Allen, all of my clips at xtream.ne.jp are provide by my QTSS in the form of streaming and fast start download.

Jeff, we will see how Murad's input works.

[Jeff Sayre]

Originally Posted by Kaku Ito

Has anyone encoded HDV to QuickTime 7 H.264 streaming like how well Apple has done? I have tried to use the preset in the Compressor 2, but the results in dropping frames often when you playback through ADSL connection or FTTH connection.

Does anyone know Apple's trick?

I have finished with testing my new encoding techniques for QTSS (QuickTime Streaming Server) and I'm very pleased with the results. You can see the befores and afters on my DVinfo.net Challenge #2 movie short submission website.

Visit http://www.sayremedia.com

You can also visit the DVInfo.net thread for my movie at http://www.dvinfo.net/conf/showthread.php?t=47873

______

Lengthy Encoding Details Follow

Please note, for the challenge postings, I did not create a H.264 version since most people were not using QT7 to view the various movie short submissions. Instead, I encoded for earlier versions of QuickTime and for Windows Media Viewer compatibility.

However, I have recently done some test encodes for H.264 encoding for QT7. I have had the same great results. The techniques I have worked out create high-quality, small file size encodes. Here are the details:

1. Unfortunately, I had the same lame results that Kaku speaks of with Apple's Compressor 2.0.1 software. It is disappointing that it produces poor results and takes forever to do so. I gave it more than a fair chance to prove itself. I tested more than 40 different encoding scenarios using the same 3-minute movie short I entered in DVC2. Each time, the results were less than satisfactory. I spent more than 40 hours testing in Compressor.

2. Then I decided to use a third party compression and encoding software package to see if I got the same or different results. I settled on Sorenson Squeeze 4.1 Compression Suite. I downloaded their demo version. It made the difference when I combined it with the other item I discovered--it is crucial to export your HDV sequence in a particular way.

3. To get great results from FCP using HDV, you must first export to QuickTime conversion (File>Export>Using QuicKTime Conversion) using the following settings:

- Make sure the format is QuickTime Movie. Name your file and choose a location to save it to.

- Click the "Options..." button next to the "Format" dropdown box. Here is where you'll tweak all the settings.

- Within the Options window, choose the "Settings..." button.

- Set "Compression Type" to HDV 1080i60 (Note: If you're wanting to encode for H.264, you would set Compression Type to H.264)

- Set "Frame Rate" to 24 fps (this could probably be 15, but since I will convert to progressive format later, I wanted to use 24 here instead)

- Set "Key Frames" to 240 (You usually use about 10 times the fps setting. Although, if you have more motion in your clips, you may want to set this lower. Try 240 first, then if that does not work reduce to 190, etcetera until you get the results you like)

- Set "Data Rate" to 500 kbits/s

- For H.264 compression type, move the "Compressor Quality" slider bar to High and make sure the "Best Quality" radio button is selected. For HDV 1080i60, there is no slider bar to set.

- Hit "Okay"

-Next, select the "Size..." button. Choose the "Use custom size" radio button and enter Width of 480, Height of 270. You do not need to go any bigger than this to create great-quality, web-based QT files. (If exporting a file for DVD use, you may want to go to the maximum TV screen size you think you will need.)

- Hit "Okay"

- Next, select the "Settings..." button under Sound. Set the "Rate" to 32 kHz or 24 kHz. The lower kHz will result in a smaller file size but the audio will still sound good on the web.

- Finally, if mastering for web-based delivery, select the "Prepare for Internet Streaming" checkbox and then select "Hinted Streaming" from the dropdown box. Next, click "Settings..." and make sure "Make Movie Self-Contained" and "Optimize Hints For Server" are selected. (Note: If you are not planning on using QTSS--QuickTime Streaming Server--then select "Fast Start" instead of "Hinted Streaming".)

- Hit "Okay" and then hit "Save."

4. Open Sorenson Squeeze and Import your newly encoded .mov export file.

5. You will batch encoded as many versions of your .mov file as you need. Squeeze allows you to batch encode just like Apple's Compressor, only it is more intuitive to use and does a much better job!

6. Under "Format & Compression Settings", you will choose which encoding algorithms you want to apply to your .mov file. For my move short, I chose algorithms from the QuickTime and MPEG-4 selections. This way, I could guarantee that people with Macs and PCs would be able to view my movie.

7. Under either QT or MP4 algorithms, you'll want to select the encoding engines that give a progressive output. That way, the interlacing artifacts are removed and you get great, smooth video on computer monitors. This is great, because you do not have to deinterlace within FCP first! (Note: If delivering for DVD on TVs, you would not select the progressive options.)

- For the QT algorithms, I used the "LG_Prog" and "Med_Prog" engines and applied them to the batch.

- For the MP4 algorithms, I used the mp4 versions of the "LG_Prog" and "Med_Prog" engines and applied them to the batch.

- For H.264, the only encoding engines available are under the MP4 algorithms. This makes sense since H.264 is a new MP4 standard. The H.264 encoding engines you choose are the AVC (Advanced Video Compression) options. For web, choose the AVC_LG_Prog and the AVC_Med_Prog engines. These are the H.264 progressive encoding engines.

8. Please note, in Squeeze, each of the encoding engines within each algorithm set can be edited. I went into each engine selection I applied to my batch and made sure that the audio was set to 24 or 22 kHz and that the engine was set to create a hinted stream. Experiment with editing the engine's settings to see what results you can obtain. Reducing the kHz of the audio helps make the file size smaller.

9. When encoding H.264 in Sorenson Squeeze, you will need to apply the "Lighten" filter to the H.264 jobs since the AVC encoding engines tend to darken the output. You'll have to tweak the filter's settings to get the results you want.


Sorenson Squeeze (http://www.sorensonmedia.com/solutions/) is what worked for me. I purchased their Compression Suite for $450. It offers me the full suite of compression products. If you know that you'll only be encoding for MP4, then you may wish to purchase Sorenson Squeeze 4.1 for MPEG-4 software for $200. It has the AVC encoding engines for H.264.

Using the detailed techniques above, I was able to take my FCP export .mov file of 450 MB in size and compress it down to 20.4 MB in size using Squeeze. View my results at http://www.sayremedia.com. I think you'll agree that the video and audio are still fantastic!

It will take experimenting on your part to see which settings work best for you. I spent more than 100 hours figuring this out and learning how to optimize QT files for web-based delivery. Be patient. What you learn through this process will prove useful to you in the future.

One final issue: all of this assumes you are creating QT for web-based delivery and viewing only. I have not used these techniques for MPEG-2 encoding for DVDs but I'm sure with just a little tweaking, I'll now be able to get great results using what I learned for Web delivery.

Good Luck!

[Kevin Shaw]

Yikes, that's a lot of steps to get to the desired output! Let us know if you find a more straightforward solution which produces acceptable results.

(In Canopus Edius I just encode directly to Windows Media HD from the timeline and I'm done.)

[Jeff Sayre]

Originally Posted by Kevin Shaw

Yikes, that's a lot of steps to get to the desired output! Let us know if you find a more straightforward solution which produces acceptable results.

(In Canopus Edius I just encode directly to Windows Media HD from the timeline and I'm done.)

Yes, it does seem lengthy, but it takes only about 20 minutes from start to finish. Most of the time is sitting around, watching the progress bar!

I gave line-by-line detail so that anyone who wanted to try, would not have difficulty. The first time or so through, it is a little confusing. However, it goes pretty quickly once you now what you're doing.

[Kaku Ito]

Jeff,

Setting the frame rate down to 15 frames with Compressor 2 made it to a reasonable outcome but it wasn't as nice as yours. I'm also having hard time converting HDV to DVD anamorphic, too. I might look into what you have tested.

Kaku

Oh, just in case you want to see the stream transcoded with Compressor 2, the it is here.

[Kaku Ito]

I forgot to thank Murad.

Thank you Murad, 15 frame trick worked great.

Kaku

[Rob Johnson]

Hi Jeff,

I was very impressed with your method and the results. I first saw this post about a week ago, at which time I visited your website and watched The Adventures of Thomas Brin series. I was wondering if you're still using the same technique as you were in '05, as outlined below?

Originally Posted by Jeff Sayre

I have finished with testing my new encoding techniques for QTSS (QuickTime Streaming Server) and I'm very pleased with the results. You can see the befores and afters on my DVinfo.net Challenge #2 movie short submission website.

Visit http://www.sayremedia.com

You can also visit the DVInfo.net thread for my movie at http://www.dvinfo.net/conf/showthread.php?t=47873

______

Lengthy Encoding Details Follow

Please note, for the challenge postings, I did not create a H.264 version since most people were not using QT7 to view the various movie short submissions. Instead, I encoded for earlier versions of QuickTime and for Windows Media Viewer compatibility.

However, I have recently done some test encodes for H.264 encoding for QT7. I have had the same great results. The techniques I have worked out create high-quality, small file size encodes. Here are the details:

1. Unfortunately, I had the same lame results that Kaku speaks of with Apple's Compressor 2.0.1 software. It is disappointing that it produces poor results and takes forever to do so. I gave it more than a fair chance to prove itself. I tested more than 40 different encoding scenarios using the same 3-minute movie short I entered in DVC2. Each time, the results were less than satisfactory. I spent more than 40 hours testing in Compressor.

2. Then I decided to use a third party compression and encoding software package to see if I got the same or different results. I settled on Sorenson Squeeze 4.1 Compression Suite. I downloaded their demo version. It made the difference when I combined it with the other item I discovered--it is crucial to export your HDV sequence in a particular way.

3. To get great results from FCP using HDV, you must first export to QuickTime conversion (File>Export>Using QuicKTime Conversion) using the following settings:

- Make sure the format is QuickTime Movie. Name your file and choose a location to save it to.

- Click the "Options..." button next to the "Format" dropdown box. Here is where you'll tweak all the settings.

- Within the Options window, choose the "Settings..." button.

- Set "Compression Type" to HDV 1080i60 (Note: If you're wanting to encode for H.264, you would set Compression Type to H.264)

- Set "Frame Rate" to 24 fps (this could probably be 15, but since I will convert to progressive format later, I wanted to use 24 here instead)

- Set "Key Frames" to 240 (You usually use about 10 times the fps setting. Although, if you have more motion in your clips, you may want to set this lower. Try 240 first, then if that does not work reduce to 190, etcetera until you get the results you like)

- Set "Data Rate" to 500 kbits/s

- For H.264 compression type, move the "Compressor Quality" slider bar to High and make sure the "Best Quality" radio button is selected. For HDV 1080i60, there is no slider bar to set.

- Hit "Okay"

-Next, select the "Size..." button. Choose the "Use custom size" radio button and enter Width of 480, Height of 270. You do not need to go any bigger than this to create great-quality, web-based QT files. (If exporting a file for DVD use, you may want to go to the maximum TV screen size you think you will need.)

- Hit "Okay"

- Next, select the "Settings..." button under Sound. Set the "Rate" to 32 kHz or 24 kHz. The lower kHz will result in a smaller file size but the audio will still sound good on the web.

- Finally, if mastering for web-based delivery, select the "Prepare for Internet Streaming" checkbox and then select "Hinted Streaming" from the dropdown box. Next, click "Settings..." and make sure "Make Movie Self-Contained" and "Optimize Hints For Server" are selected. (Note: If you are not planning on using QTSS--QuickTime Streaming Server--then select "Fast Start" instead of "Hinted Streaming".)

- Hit "Okay" and then hit "Save."

4. Open Sorenson Squeeze and Import your newly encoded .mov export file.

5. You will batch encoded as many versions of your .mov file as you need. Squeeze allows you to batch encode just like Apple's Compressor, only it is more intuitive to use and does a much better job!

6. Under "Format & Compression Settings", you will choose which encoding algorithms you want to apply to your .mov file. For my move short, I chose algorithms from the QuickTime and MPEG-4 selections. This way, I could guarantee that people with Macs and PCs would be able to view my movie.

7. Under either QT or MP4 algorithms, you'll want to select the encoding engines that give a progressive output. That way, the interlacing artifacts are removed and you get great, smooth video on computer monitors. This is great, because you do not have to deinterlace within FCP first! (Note: If delivering for DVD on TVs, you would not select the progressive options.)

- For the QT algorithms, I used the "LG_Prog" and "Med_Prog" engines and applied them to the batch.

- For the MP4 algorithms, I used the mp4 versions of the "LG_Prog" and "Med_Prog" engines and applied them to the batch.

- For H.264, the only encoding engines available are under the MP4 algorithms. This makes sense since H.264 is a new MP4 standard. The H.264 encoding engines you choose are the AVC (Advanced Video Compression) options. For web, choose the AVC_LG_Prog and the AVC_Med_Prog engines. These are the H.264 progressive encoding engines.

8. Please note, in Squeeze, each of the encoding engines within each algorithm set can be edited. I went into each engine selection I applied to my batch and made sure that the audio was set to 24 or 22 kHz and that the engine was set to create a hinted stream. Experiment with editing the engine's settings to see what results you can obtain. Reducing the kHz of the audio helps make the file size smaller.

9. When encoding H.264 in Sorenson Squeeze, you will need to apply the "Lighten" filter to the H.264 jobs since the AVC encoding engines tend to darken the output. You'll have to tweak the filter's settings to get the results you want.


Sorenson Squeeze (http://www.sorensonmedia.com/solutions/) is what worked for me. I purchased their Compression Suite for $450. It offers me the full suite of compression products. If you know that you'll only be encoding for MP4, then you may wish to purchase Sorenson Squeeze 4.1 for MPEG-4 software for $200. It has the AVC encoding engines for H.264.

Using the detailed techniques above, I was able to take my FCP export .mov file of 450 MB in size and compress it down to 20.4 MB in size using Squeeze. View my results at http://www.sayremedia.com. I think you'll agree that the video and audio are still fantastic!

It will take experimenting on your part to see which settings work best for you. I spent more than 100 hours figuring this out and learning how to optimize QT files for web-based delivery. Be patient. What you learn through this process will prove useful to you in the future.

One final issue: all of this assumes you are creating QT for web-based delivery and viewing only. I have not used these techniques for MPEG-2 encoding for DVDs but I'm sure with just a little tweaking, I'll now be able to get great results using what I learned for Web delivery.

Good Luck!

[Michael Blieden]

I encoded some files a few days ago using H.264 and got wonderful results. These are HDV files exported to a 700x394 frame size, and I set the data rate restriction to 1688kb. The resultes were perfect for what I needed. In the last few days, however, regardless of what number I enter into the data rate field, I get files with a data rate in excess of 3,000kb. These files are too big to be useful, and I don't know why compressor won't honor my data rate restriction. Anyone have any tips?

[Jeff Mack]

Kaku,

I just saw your post. I have been posting a lot of HDV up on the web in H.264. Check out the clips here: www.texasmusiclegends.net

I simply took my sequence and exported it to QT7 compatable and then chose H.264 800KBS Streaming. It is hinted streaming. Not really sure what that does. When I got the inspector open, I set the size to custom 960x540 16x9.

Takes a little while but it seems to work for me.

Others, let me know if I am missing something in my workflow.

Jeff