Cineform and multiple CPU utilization poor? at DVinfo.net

Marty Hudzik · December 4th, 2011, 06:54 PM

I am editing on a PC with a brand new 6 core Intel Processor. I am using a Cineform 1080P 23.97 preset in Premiere and I import some ProRes clips. When I drop them on the timeline, add a few effects to them and render a preview, it is taking a really long time. I check task manager and the CPU usage is 15%. All 12 CPU threads are barely raised.

I copy this clip and drop it into a standard HDV 1080 timeline and build a preview and it goes 4 times faster and utilizes 50% of the CPU...again, all 12 cores are still doing very little each, but beating Cineform by 4x.

So....is this by design? I realize Cineform is a higher quality codec, and may require additional processing power to render, but at this moment, it's not even using "any" of the horsepower of my system to do this. I might as well have a dual core. At least Adobe HDV preset utilizes "half" of the power.

So....can anyone shed some light on this scenario? I have owned Cineform is some facet for 5 years and I have the full version of NEO. I just really came back to using Cineform as I am now shooting with a Atomos Samurai and acquiring 4-2-2 footage in Pro-Res Codec. Since I am on a PC, I am choosing to export and edit using Cineform presets as to retain the additional color information and not add compression unneccesarily.

Thanks for any help.

David Newman · December 4th, 2011, 08:04 PM

The preview render, not the decoding, might be slower for a number of reasons but it not the speed of the codec itself. If you are unconcerned with preview quality you can switch to using I-frame MPEG. It is not a faster codec, but Premeire doesn't get so much in the way (we find the Adobe API for third parties takes 80% of the time -- we mentioned it to them in the past.)

Marty Hudzik · December 4th, 2011, 11:02 PM

David,
I find that it also affects the output timing too. A 30 second clip with a few effects on it renders out of premiere in 1:30 to h.264 but takes over 4 minutes to go out to cineform YUV 4:2:2. Is this normal? The file has originated as a Apple ProRes quicktime at the 422 setting. Perhaps that plays in?

Regardless, I am still only seeing 15-16% CPU usage with Cineform export whereas I see 55-65% consistently with Adobe h.264 export, which is about the same as when I was building previews in the timeline.

Again....is this normal? I don't want to beat my head against the wall if this is just the cost of working in Cineform? I'm not saying I won't use Cineform, but it would stink to be force to use only 15% of my CPU capacity.

Oh...if this helps at all, when I export to uncompressed Avi the speed is the same as going to h.264, so the cineform encoding seems to be at least playing a part in the bottleneck.

Carlo Macchiavello · December 9th, 2011, 03:32 AM

the bottleneck is to not use cineform like master...
if you put prores on timeline, you not use the cineform acceleration.
if you work with compressed video like prores, premiere work two time to decompress and recompress in cineform, if you work on a cineform timeline, with cineform video (the goal is to conform all to cineform codec) you can see a speed up.
premiere is slow, and if you work with mixed codec timeline you can add another bottleneck.
the idea of cineform is to be a DI Digital intermediate. if you convert prores in cineform with right compression you have a bigger file, but it need less cpu work to read and decompress it.

John Quandt · December 9th, 2011, 11:11 PM

What I think you've discovered is that Core i7 processors are so fast that they're waiting for data from the hard drive. Your ssytem is disc I/O limited. When you encode video into Cineform format, it's typically four times larger than the original compressed format. Cineform is much easier for the cpu to edit because it's a continuous format like DV instead of the i-frame plus multiple difference frames of codecs like HDV and AVCHD. To increase the throughput when editing Cineform, you'll need a faster hard drive system to handle the extra data.

Marty Hudzik · December 11th, 2011, 07:19 AM

I am not buying that this is a hard drive throughput issue. It has something to do with encoding and optimization for multiple cores. I have used Cineform for a long time. With the effects that I have applied, their is no way the hard drive is the bottleneck. If I create a timeline as an HDV timeline and drop PRoRes footage on it and export to h.264 it uses 50% CPU. If I create a new timeline and make it CIneform and apply the exact same clips and effects and export to Cineform it uses 15% CPU to encode. So, the bottle neck is in the encoder. It is just frustrating to have invested in a 6core-12thread CPU to see only 15% get used for rendering. I might as well have not upgraded if I cannot resolve this.

I am also trying to test converting files to Cineform with HDlink and editing on a Cineform timeline, but my first few conversions actually degraded my ProRes footage before I ever got it into Premiere which seems odd. I chose the Cineform HQ setting and the files went up in size by about 10% but the resulting footage looked brighter, had lost contrast and I swear I could see compression artifacts in the footage that were not there originally. Is this possible?

Roger Averdahl · December 11th, 2011, 11:01 AM

Quote:

Originally Posted by Marty Hudzik

I am not buying that this is a hard drive throughput issue.

I am with you Marty, i don't buy that either.

I once tried it and increased the disk I/O with 300% when i used a RAID0 but the gain in render times was only 5%. I had the media on one SSD that gave an I/O of +230MB/s and the output was set to a RAID0 that had an I/O of approx +350MB/s. I tried the reversed as well, having the source media on the RAID0 and set the output to the SSD or back to the RAID0.

Since copying the CineForm file from disk X to Y was/is lightning fast there must be something else than disk I/O alone. David has already mentioned the Adobe API in this thread and that's what we should blame, not disk I/O alone.

Jay Bloomfield · December 14th, 2011, 01:24 PM

A codec is something that both encodes and decodes. Encoding is the most challenging and that's why that part of a codec is efficiently multithreaded, either using the CPU or the GPU. However, the decoding part usually receives less attention these days, since the frame rate that the screen can display is usually the limiting factor. There is no benefit in decoding at 800 fps, if the maximum that you can display is 60 fps. The flaw in that logic is that sometimes, you are not displaying to an output device, but rather you are decoding, adding in various FX and then encoding. The latter two items may be efficiently multithreaded, but the decoding will be the bottleneck. Maybe that's what David was saying above.

December 4th, 2011, 06:54 PM	#1
Marty Hudzik Trustee Join Date: Jul 2004 Location: Canton, Ohio Posts: 1,771	Cineform and multiple CPU utilization poor? I am editing on a PC with a brand new 6 core Intel Processor. I am using a Cineform 1080P 23.97 preset in Premiere and I import some ProRes clips. When I drop them on the timeline, add a few effects to them and render a preview, it is taking a really long time. I check task manager and the CPU usage is 15%. All 12 CPU threads are barely raised. I copy this clip and drop it into a standard HDV 1080 timeline and build a preview and it goes 4 times faster and utilizes 50% of the CPU...again, all 12 cores are still doing very little each, but beating Cineform by 4x. So....is this by design? I realize Cineform is a higher quality codec, and may require additional processing power to render, but at this moment, it's not even using "any" of the horsepower of my system to do this. I might as well have a dual core. At least Adobe HDV preset utilizes "half" of the power. So....can anyone shed some light on this scenario? I have owned Cineform is some facet for 5 years and I have the full version of NEO. I just really came back to using Cineform as I am now shooting with a Atomos Samurai and acquiring 4-2-2 footage in Pro-Res Codec. Since I am on a PC, I am choosing to export and edit using Cineform presets as to retain the additional color information and not add compression unneccesarily. Thanks for any help.

December 4th, 2011, 08:04 PM	#2
David Newman CTO, CineForm Inc. Join Date: Jul 2003 Location: Cardiff-by-the-Sea, California Posts: 8,095	Re: Cineform and multiple CPU utilization poor? The preview render, not the decoding, might be slower for a number of reasons but it not the speed of the codec itself. If you are unconcerned with preview quality you can switch to using I-frame MPEG. It is not a faster codec, but Premeire doesn't get so much in the way (we find the Adobe API for third parties takes 80% of the time -- we mentioned it to them in the past.) __________________ David Newman -- web: www.gopro.com blog: cineform.blogspot.com -- twitter: twitter.com/David_Newman

December 4th, 2011, 11:02 PM	#3
Marty Hudzik Trustee Join Date: Jul 2004 Location: Canton, Ohio Posts: 1,771	Re: Cineform and multiple CPU utilization poor? David, I find that it also affects the output timing too. A 30 second clip with a few effects on it renders out of premiere in 1:30 to h.264 but takes over 4 minutes to go out to cineform YUV 4:2:2. Is this normal? The file has originated as a Apple ProRes quicktime at the 422 setting. Perhaps that plays in? Regardless, I am still only seeing 15-16% CPU usage with Cineform export whereas I see 55-65% consistently with Adobe h.264 export, which is about the same as when I was building previews in the timeline. Again....is this normal? I don't want to beat my head against the wall if this is just the cost of working in Cineform? I'm not saying I won't use Cineform, but it would stink to be force to use only 15% of my CPU capacity. Oh...if this helps at all, when I export to uncompressed Avi the speed is the same as going to h.264, so the cineform encoding seems to be at least playing a part in the bottleneck.

December 9th, 2011, 03:32 AM	#4
Carlo Macchiavello Regular Crew Join Date: Oct 2010 Location: italy Posts: 107	Re: Cineform and multiple CPU utilization poor? the bottleneck is to not use cineform like master... if you put prores on timeline, you not use the cineform acceleration. if you work with compressed video like prores, premiere work two time to decompress and recompress in cineform, if you work on a cineform timeline, with cineform video (the goal is to conform all to cineform codec) you can see a speed up. premiere is slow, and if you work with mixed codec timeline you can add another bottleneck. the idea of cineform is to be a DI Digital intermediate. if you convert prores in cineform with right compression you have a bigger file, but it need less cpu work to read and decompress it.

December 9th, 2011, 11:11 PM	#5
John Quandt Regular Crew Join Date: May 2009 Location: Ogden UT Posts: 40	Re: Cineform and multiple CPU utilization poor? What I think you've discovered is that Core i7 processors are so fast that they're waiting for data from the hard drive. Your ssytem is disc I/O limited. When you encode video into Cineform format, it's typically four times larger than the original compressed format. Cineform is much easier for the cpu to edit because it's a continuous format like DV instead of the i-frame plus multiple difference frames of codecs like HDV and AVCHD. To increase the throughput when editing Cineform, you'll need a faster hard drive system to handle the extra data.

December 11th, 2011, 07:19 AM	#6
Marty Hudzik Trustee Join Date: Jul 2004 Location: Canton, Ohio Posts: 1,771	Re: Cineform and multiple CPU utilization poor? I am not buying that this is a hard drive throughput issue. It has something to do with encoding and optimization for multiple cores. I have used Cineform for a long time. With the effects that I have applied, their is no way the hard drive is the bottleneck. If I create a timeline as an HDV timeline and drop PRoRes footage on it and export to h.264 it uses 50% CPU. If I create a new timeline and make it CIneform and apply the exact same clips and effects and export to Cineform it uses 15% CPU to encode. So, the bottle neck is in the encoder. It is just frustrating to have invested in a 6core-12thread CPU to see only 15% get used for rendering. I might as well have not upgraded if I cannot resolve this. I am also trying to test converting files to Cineform with HDlink and editing on a Cineform timeline, but my first few conversions actually degraded my ProRes footage before I ever got it into Premiere which seems odd. I chose the Cineform HQ setting and the files went up in size by about 10% but the resulting footage looked brighter, had lost contrast and I swear I could see compression artifacts in the footage that were not there originally. Is this possible?

December 14th, 2011, 01:24 PM	#8
Jay Bloomfield Major Player Join Date: Feb 2008 Location: Voorheesville, NY Posts: 433	Re: Cineform and multiple CPU utilization poor? A codec is something that both encodes and decodes. Encoding is the most challenging and that's why that part of a codec is efficiently multithreaded, either using the CPU or the GPU. However, the decoding part usually receives less attention these days, since the frame rate that the screen can display is usually the limiting factor. There is no benefit in decoding at 800 fps, if the maximum that you can display is 60 fps. The flaw in that logic is that sometimes, you are not displaying to an output device, but rather you are decoding, adding in various FX and then encoding. The latter two items may be efficiently multithreaded, but the decoding will be the bottleneck. Maybe that's what David was saying above.