What to upgrade next in the computer? at DVinfo.net

Benjamin Maas · October 22nd, 2011, 11:39 AM

So A year ago when I was getting into this, I didn't expect to be working with more than one or two video streams at a time. Basic work. The video is AVCHD off a Sony camera.

Now, things have changed quite a bit and my last edit was causing my computer to beg for mercy. I had 4 cameras that I was editing in the multicam screen, all had color correction and other processing going on. There was a lot of split screen stuff, crop and zoom, sharpening, etc... The computer started behaving really badly- dropping tons of frames and big glitches, playback wouldn't start or end in the right place. I had this strange "looping" problem where every spacebar push restarted the video from the same point. It got to a point where there were so many problems that to get through 30 seconds of material could take the better part of a half hour. And timing exact cuts between cams? Forget it.

My computer is as follows a pretty good machine. Intel 980x processor, Asus P6X58 Mobo, EVGA Nvidia GTX 470 (subsequently added a Gforce 8400GS to get an extra HDMI out for my TV), 12 GB of RAM, a 2 TB RAID0 SATAIII work drive and a 1 TB (non raid) render drive.

So the question is- what can I do to improve upon the setup. Ideas have included, more memory (perhaps doubling it), a Matrox MXO2LE Max, better video card perhaps.,... Trying not to spend huge cash as the video side of my business is barely breaking even, but this work is killing me.

Another option is avoiding the hardware change and going to encoding all my video as Cineform or some other intermediate codec to hopefully save on some processing.

What will give me the most mileage? What can I do with my workflow to make all of this run a bit more smoothly? Ideas?

Thanks!

--Ben

Bart Walczak · October 22nd, 2011, 12:28 PM

Benjamin, if you only have a single drive in a RAID0 array, then there's no gain of speed at all, it is like you would have a single drive. Unless I'm misunderstanding something. You could add drives to your RAID array to get better throughput, which might help you.

However I would advise you first to do some diagnostics - run Task Manager and see if your cores all at 100% when playing back the video streams. If so, then unfortunately you'd have to upgrade your CPU, which means probably also the mainboard. Or attempt to overclock, but this is a risky business, and with 980X there will be not very much to gain as well. However if the cores are not maxed out, then there is little to be gained here.

Also, check what is your disk data rate. Run HDTach or other diagnostic program to find out, and then find out what is the total data rate of your streams (AVCHD usually has about 24 mbps = 3 MB/s, which is pretty small, and any decent drive should be able to handle 4 streams without much problems). Run a defragmenting utility and check, if your drive is not too fragmented.

You could think about increasing RAM to 16 or 24 GBs, but it won't be much of a difference here I think

Benjamin Maas · October 22nd, 2011, 09:39 PM

RAID 0 by its nature is 2 drives that are striped. In my case, using the onboard RAID controller in the motherboard... I have two identical 1 TB drives that are striped to make a single drive (as the computer sees it). Each drive is a 6.0 GB/s drive so the work drive is rather fast.

I would think that if the cores aren't balancing that is an Adobe issue... Am I incorrect? With 3 streams, I seem to be ok, but once that 4th camera is added, that is where I start having issues.

--Ben

Bart Walczak · October 23rd, 2011, 11:46 AM

Adobe is really good at maximizing all cores use, even in 6-core CPUs I believe. Did you try to reduce the playback resolution in player window?

The fact that a drive uses 6GB/s SATA interface does not yet mean anything about its actual speed, it is only the limit of the interface. I would suggest running a HDTach or other application to see what kind of bandwidth you actually have. However, the problem might actually be the number of reads and writes, not the bandwidth. With 4 AVCHD streams you would hardly reach 30 MB/s, which any decent drive should be able to handle easily. If your drive is not fragmented, and not too full (NTFS drives have a tendency to slow down when more than 80% full), it should be able to handle it with little problems.

What I would test myself:
- convert a little of AVCHD footage to something less CPU-intensive, like DVCPRO HD 24p (although it's 40 mbps) or even MPEG-2 with 24 mbps, to see if it helps - then the issue is indeed the power of your CPU.
- convert a little of the footage to lower bitrate, like 8-9, and see if it helps - if it does, then it is most likely HDD, either too fragmented (run defrag first), or too full, or simply not able to keep up with the random reads.

Mind you, these are just tests to establish what is the weak spot here, I don't suggest you edit with either kind of converted footage :)

Randy Painter · October 23rd, 2011, 06:38 PM

Sounds like your reading and writing to the same disk, correct? Or am I reading this wrong? Always use a scratch disk to render to, whether another raid setup or single. Myself, I render to a ssd drive.
Check your virtual memory to see if it's full. My guess is your thrashing your drive when the virtual memory gets full and has to write to disk, which slows down your system.

Benjamin Maas · October 24th, 2011, 10:53 AM

I am not rendering to the same drive... As you'll see in what I wrote earlier- I have 2 TB for work in a RAID 0 configuration. I also have a 1 TB render drive (a single non-raid SATA III disc).

I'll check what's going on with my memory, though...

--Ben

Randy Painter · October 24th, 2011, 09:41 PM

Sorry, missed that. Didn't see what OS your using. If your paging out, then more disk reads/writes. Ever thought about a raptor drive for tour render drive? Noisy but fast. Also, the larger the drive, the more platters to write to. Best check the memory first to see if your maxing it out before going further.

Benjamin Maas · October 24th, 2011, 10:07 PM

So I just did some of the standard Windows diagnostics (using Win 7-64 here)...

The results were all over the place. Memory was hung out at 6GB most of the time and peaked as high as 8. I've got Premiere set up to take up to 9GB, so there should be plenty of overhead there as my reading was comprehensive including windows.

Disc was also all over the place- topping out at 27-30 MB/sec which once again is well within what the discs should handle. The setting for percentage at max usage in the Resource Manager said as high as 2%.

CPU did max at 100% or above a couple times. Looking at the core usage in the Task Manager shows that most of the time the CPU usage is pretty low, but that it does have a tendency to spike rather high. Also, Cores 10 and 11 were sometimes near 0% when all of the others were nearly maxed. (Hex core 980x processor- doubled in the task manager)

The strange thing, though, is that even when all of the various aspects of the computer seem low, it is still dropping frames upon playback. For a lot of the time, it isn't a major problem, but I edit music most of the time and trying to time camera changes when frames are being dropped can be rough

So.... Assuming that CPU is the main culprit here. I don't have the bucks to upgrade this computer as it is pretty high-end already. A CPU upgrade would essentially mean a whole new machine which would run several thousand dollars. Are there workflow things I can do that would help. I'm working on these AVCHD files as they come off the camera most of the time. Would an intermediate codec help perhaps? (ie Cineform) I've already modified some of the sessions to get rid of camera angles that aren't going to work and it seems to help some. It seems like 3 video streams will generally work pretty well, but 4 starts to give me problems.

Thanks!

--Ben

Harm Millaard · October 25th, 2011, 02:13 AM

Benjamin,

If you could run the PPBM5 Benchmark and submit your data, I may be able to identify where your bottlenecks are. One thing is certain, clock speed is very important while editing AVCHD and with your unlocked CPU it is relatively easy to overclock (but time consuming to achieve stability). Increasing memory to 24 GB will help as well, but if your disk setup is lacking or your system is not properly tuned, the benefits of adding memory are negligent. You can find a lot of tuning tips and disk configurations on the Adobe Premiere Pro Hardware forum, for instance under the Overview tab and the FAQ section.

Benjamin Maas · October 25th, 2011, 11:24 PM

Harm-

Thanks for the link to the Benchmark. I've got stuff rendering right now and then I'm going out of town, but as soon as I get back on Friday, I'll give it a run.

Something seems to be wrong, though... A concert that I recorded- single camera, documentary recording would normally take me about 2 hours or so to render. This is for about 90-100 min of footage rendering to Mpeg 2 for a DVD from 720P. Today, it took over 5 hours to do the same thing.

Looks like there may be multiple issues.

--Ben

Bo Skelmose · October 26th, 2011, 12:07 AM

Hi
I have a Asus P6WS-pro x58 motherboard with 980X cpu, 24 gB ram and a Nvidia GTX 470. From time to time it can drop frames too.
I have a Highpoint Sata raid controller with 12 fast drives in a raid. (not 0 but 5 (I think)) Throuhput is very fast but maybe the raid 5 gives some kind of lack in response. I have 4 drives in raid 0 on the motherboard - they have the same problem though.
I have not experienced any difference from 12gB to 24gB - but I have not measured it.

Benjamin Maas · October 26th, 2011, 10:47 AM

Ran the benchmark and submitted the results. I'm interested to hear if there is anything of interest noticed in them.

Yesterday's renders were just painful. Took over 7 hours for 2 projects to render.

--Ben

Randall Leong · October 26th, 2011, 11:06 AM

Did you consider the possibility that you might have a conflict between the GTX 470 and the 8400 GS? Those two GPUs are of totally different generations. And that might have forced the software-only MPE mode even though you had GPU accelerated mode enabled.

By the way, I can't wait long enough for those results to be posted on the PPBM5 results list.

Harm Millaard · October 26th, 2011, 12:17 PM

Ben,

Thanks for your submission. I'll process them tomorrow and give you feedback then.

Harm Millaard · October 27th, 2011, 02:12 AM

Ben,

I processed your results and you are at this moment at rank # 94.

What could be improved to get better results and what is already very good?

Your MPE results with the GTX 470 are very good. The most disappointing result with CS5.03 is your MPEG2 result with 64 seconds. My guess is that you can shave off 20 to 30 seconds or even more from that score by increasing memory from 12 to 24 GB and overclocking your CPU to around 4.0 GHz. That will also improve your H.264 score, because it is very much clock speed dependent. Adding an extra disk to create a second raid0 instead of a single disk as you currently have can improve your disk results by a few seconds.

Since overclocking is free, that is what I would look at first. Second is memory and then finally an extra disk.

October 22nd, 2011, 11:39 AM	#1
Benjamin Maas Regular Crew Join Date: Jul 2010 Location: Long Beach, CA Posts: 150	What to upgrade next in the computer? So A year ago when I was getting into this, I didn't expect to be working with more than one or two video streams at a time. Basic work. The video is AVCHD off a Sony camera. Now, things have changed quite a bit and my last edit was causing my computer to beg for mercy. I had 4 cameras that I was editing in the multicam screen, all had color correction and other processing going on. There was a lot of split screen stuff, crop and zoom, sharpening, etc... The computer started behaving really badly- dropping tons of frames and big glitches, playback wouldn't start or end in the right place. I had this strange "looping" problem where every spacebar push restarted the video from the same point. It got to a point where there were so many problems that to get through 30 seconds of material could take the better part of a half hour. And timing exact cuts between cams? Forget it. My computer is as follows a pretty good machine. Intel 980x processor, Asus P6X58 Mobo, EVGA Nvidia GTX 470 (subsequently added a Gforce 8400GS to get an extra HDMI out for my TV), 12 GB of RAM, a 2 TB RAID0 SATAIII work drive and a 1 TB (non raid) render drive. So the question is- what can I do to improve upon the setup. Ideas have included, more memory (perhaps doubling it), a Matrox MXO2LE Max, better video card perhaps.,... Trying not to spend huge cash as the video side of my business is barely breaking even, but this work is killing me. Another option is avoiding the hardware change and going to encoding all my video as Cineform or some other intermediate codec to hopefully save on some processing. What will give me the most mileage? What can I do with my workflow to make all of this run a bit more smoothly? Ideas? Thanks! --Ben __________________ Benjamin Maas • Fifth Circle Audio • Signal Hill, CA http://www.fifthcircle.com

October 22nd, 2011, 12:28 PM	#2
Bart Walczak Major Player Join Date: Apr 2006 Location: Warsaw/Poland Posts: 716	Re: What to upgrade next in the computer? Benjamin, if you only have a single drive in a RAID0 array, then there's no gain of speed at all, it is like you would have a single drive. Unless I'm misunderstanding something. You could add drives to your RAID array to get better throughput, which might help you. However I would advise you first to do some diagnostics - run Task Manager and see if your cores all at 100% when playing back the video streams. If so, then unfortunately you'd have to upgrade your CPU, which means probably also the mainboard. Or attempt to overclock, but this is a risky business, and with 980X there will be not very much to gain as well. However if the cores are not maxed out, then there is little to be gained here. Also, check what is your disk data rate. Run HDTach or other diagnostic program to find out, and then find out what is the total data rate of your streams (AVCHD usually has about 24 mbps = 3 MB/s, which is pretty small, and any decent drive should be able to handle 4 streams without much problems). Run a defragmenting utility and check, if your drive is not too fragmented. You could think about increasing RAM to 16 or 24 GBs, but it won't be much of a difference here I think __________________ Creative Impatience - The Solace of Simple Solutions. A few useful plugins for Adobe users, and my remarks on the tools and the craft in general.

October 22nd, 2011, 09:39 PM	#3
Benjamin Maas Regular Crew Join Date: Jul 2010 Location: Long Beach, CA Posts: 150	Re: What to upgrade next in the computer? RAID 0 by its nature is 2 drives that are striped. In my case, using the onboard RAID controller in the motherboard... I have two identical 1 TB drives that are striped to make a single drive (as the computer sees it). Each drive is a 6.0 GB/s drive so the work drive is rather fast. I would think that if the cores aren't balancing that is an Adobe issue... Am I incorrect? With 3 streams, I seem to be ok, but once that 4th camera is added, that is where I start having issues. --Ben __________________ Benjamin Maas • Fifth Circle Audio • Signal Hill, CA http://www.fifthcircle.com

October 23rd, 2011, 11:46 AM	#4
Bart Walczak Major Player Join Date: Apr 2006 Location: Warsaw/Poland Posts: 716	Re: What to upgrade next in the computer? Adobe is really good at maximizing all cores use, even in 6-core CPUs I believe. Did you try to reduce the playback resolution in player window? The fact that a drive uses 6GB/s SATA interface does not yet mean anything about its actual speed, it is only the limit of the interface. I would suggest running a HDTach or other application to see what kind of bandwidth you actually have. However, the problem might actually be the number of reads and writes, not the bandwidth. With 4 AVCHD streams you would hardly reach 30 MB/s, which any decent drive should be able to handle easily. If your drive is not fragmented, and not too full (NTFS drives have a tendency to slow down when more than 80% full), it should be able to handle it with little problems. What I would test myself: - convert a little of AVCHD footage to something less CPU-intensive, like DVCPRO HD 24p (although it's 40 mbps) or even MPEG-2 with 24 mbps, to see if it helps - then the issue is indeed the power of your CPU. - convert a little of the footage to lower bitrate, like 8-9, and see if it helps - if it does, then it is most likely HDD, either too fragmented (run defrag first), or too full, or simply not able to keep up with the random reads. Mind you, these are just tests to establish what is the weak spot here, I don't suggest you edit with either kind of converted footage :) __________________ Creative Impatience - The Solace of Simple Solutions. A few useful plugins for Adobe users, and my remarks on the tools and the craft in general.

October 23rd, 2011, 06:38 PM	#5
Randy Painter Regular Crew Join Date: Jun 2006 Location: st cloud, Mn Posts: 79	Re: What to upgrade next in the computer? Sounds like your reading and writing to the same disk, correct? Or am I reading this wrong? Always use a scratch disk to render to, whether another raid setup or single. Myself, I render to a ssd drive. Check your virtual memory to see if it's full. My guess is your thrashing your drive when the virtual memory gets full and has to write to disk, which slows down your system.

October 24th, 2011, 10:53 AM	#6
Benjamin Maas Regular Crew Join Date: Jul 2010 Location: Long Beach, CA Posts: 150	Re: What to upgrade next in the computer? I am not rendering to the same drive... As you'll see in what I wrote earlier- I have 2 TB for work in a RAID 0 configuration. I also have a 1 TB render drive (a single non-raid SATA III disc). I'll check what's going on with my memory, though... --Ben __________________ Benjamin Maas • Fifth Circle Audio • Signal Hill, CA http://www.fifthcircle.com

October 24th, 2011, 09:41 PM	#7
Randy Painter Regular Crew Join Date: Jun 2006 Location: st cloud, Mn Posts: 79	Re: What to upgrade next in the computer? Sorry, missed that. Didn't see what OS your using. If your paging out, then more disk reads/writes. Ever thought about a raptor drive for tour render drive? Noisy but fast. Also, the larger the drive, the more platters to write to. Best check the memory first to see if your maxing it out before going further.

October 24th, 2011, 10:07 PM	#8
Benjamin Maas Regular Crew Join Date: Jul 2010 Location: Long Beach, CA Posts: 150	Re: What to upgrade next in the computer? So I just did some of the standard Windows diagnostics (using Win 7-64 here)... The results were all over the place. Memory was hung out at 6GB most of the time and peaked as high as 8. I've got Premiere set up to take up to 9GB, so there should be plenty of overhead there as my reading was comprehensive including windows. Disc was also all over the place- topping out at 27-30 MB/sec which once again is well within what the discs should handle. The setting for percentage at max usage in the Resource Manager said as high as 2%. CPU did max at 100% or above a couple times. Looking at the core usage in the Task Manager shows that most of the time the CPU usage is pretty low, but that it does have a tendency to spike rather high. Also, Cores 10 and 11 were sometimes near 0% when all of the others were nearly maxed. (Hex core 980x processor- doubled in the task manager) The strange thing, though, is that even when all of the various aspects of the computer seem low, it is still dropping frames upon playback. For a lot of the time, it isn't a major problem, but I edit music most of the time and trying to time camera changes when frames are being dropped can be rough So.... Assuming that CPU is the main culprit here. I don't have the bucks to upgrade this computer as it is pretty high-end already. A CPU upgrade would essentially mean a whole new machine which would run several thousand dollars. Are there workflow things I can do that would help. I'm working on these AVCHD files as they come off the camera most of the time. Would an intermediate codec help perhaps? (ie Cineform) I've already modified some of the sessions to get rid of camera angles that aren't going to work and it seems to help some. It seems like 3 video streams will generally work pretty well, but 4 starts to give me problems. Thanks! --Ben __________________ Benjamin Maas • Fifth Circle Audio • Signal Hill, CA http://www.fifthcircle.com

October 25th, 2011, 02:13 AM	#9
Harm Millaard Trustee Join Date: Aug 2006 Location: Rotterdam, Netherlands Posts: 1,832	Re: What to upgrade next in the computer? Benjamin, If you could run the PPBM5 Benchmark and submit your data, I may be able to identify where your bottlenecks are. One thing is certain, clock speed is very important while editing AVCHD and with your unlocked CPU it is relatively easy to overclock (but time consuming to achieve stability). Increasing memory to 24 GB will help as well, but if your disk setup is lacking or your system is not properly tuned, the benefits of adding memory are negligent. You can find a lot of tuning tips and disk configurations on the Adobe Premiere Pro Hardware forum, for instance under the Overview tab and the FAQ section.

October 25th, 2011, 11:24 PM	#10
Benjamin Maas Regular Crew Join Date: Jul 2010 Location: Long Beach, CA Posts: 150	Re: What to upgrade next in the computer? Harm- Thanks for the link to the Benchmark. I've got stuff rendering right now and then I'm going out of town, but as soon as I get back on Friday, I'll give it a run. Something seems to be wrong, though... A concert that I recorded- single camera, documentary recording would normally take me about 2 hours or so to render. This is for about 90-100 min of footage rendering to Mpeg 2 for a DVD from 720P. Today, it took over 5 hours to do the same thing. Looks like there may be multiple issues. --Ben __________________ Benjamin Maas • Fifth Circle Audio • Signal Hill, CA http://www.fifthcircle.com

October 26th, 2011, 12:07 AM	#11
Bo Skelmose Major Player Join Date: Apr 2008 Location: Denmark Posts: 495	Re: What to upgrade next in the computer? Hi I have a Asus P6WS-pro x58 motherboard with 980X cpu, 24 gB ram and a Nvidia GTX 470. From time to time it can drop frames too. I have a Highpoint Sata raid controller with 12 fast drives in a raid. (not 0 but 5 (I think)) Throuhput is very fast but maybe the raid 5 gives some kind of lack in response. I have 4 drives in raid 0 on the motherboard - they have the same problem though. I have not experienced any difference from 12gB to 24gB - but I have not measured it.

October 26th, 2011, 10:47 AM	#12
Benjamin Maas Regular Crew Join Date: Jul 2010 Location: Long Beach, CA Posts: 150	Re: What to upgrade next in the computer? Ran the benchmark and submitted the results. I'm interested to hear if there is anything of interest noticed in them. Yesterday's renders were just painful. Took over 7 hours for 2 projects to render. --Ben __________________ Benjamin Maas • Fifth Circle Audio • Signal Hill, CA http://www.fifthcircle.com

October 26th, 2011, 11:06 AM	#13
Randall Leong Major Player Join Date: Nov 2009 Location: Melrose Park, Illinois, USA Posts: 936	Re: What to upgrade next in the computer? Did you consider the possibility that you might have a conflict between the GTX 470 and the 8400 GS? Those two GPUs are of totally different generations. And that might have forced the software-only MPE mode even though you had GPU accelerated mode enabled. By the way, I can't wait long enough for those results to be posted on the PPBM5 results list.

October 26th, 2011, 12:17 PM	#14
Harm Millaard Trustee Join Date: Aug 2006 Location: Rotterdam, Netherlands Posts: 1,832	Re: What to upgrade next in the computer? Ben, Thanks for your submission. I'll process them tomorrow and give you feedback then.

October 27th, 2011, 02:12 AM	#15
Harm Millaard Trustee Join Date: Aug 2006 Location: Rotterdam, Netherlands Posts: 1,832	Re: What to upgrade next in the computer? Ben, I processed your results and you are at this moment at rank # 94. What could be improved to get better results and what is already very good? Your MPE results with the GTX 470 are very good. The most disappointing result with CS5.03 is your MPEG2 result with 64 seconds. My guess is that you can shave off 20 to 30 seconds or even more from that score by increasing memory from 12 to 24 GB and overclocking your CPU to around 4.0 GHz. That will also improve your H.264 score, because it is very much clock speed dependent. Adding an extra disk to create a second raid0 instead of a single disk as you currently have can improve your disk results by a few seconds. Since overclocking is free, that is what I would look at first. Second is memory and then finally an extra disk.