Network setup for workgroup editing

Harm Millaard · November 8th, 2009, 02:11 PM

Stephen,

I just heard from my son. His comments:

I can only give you good advise if I know the budget…

In an ideal situation they would go for a Netapp storage solution with snapshotting at the block level for backup, so older files can be restored instantly. Such equipment starts at around € 20,000 and may run into the millions... Look here: http://www.netapp.com/us/products/st...ech-specs.html

I can't really give an advise without knowing the budget available, but I do think that fiber channel is the best solution by far, but again, the equipment to use is largely budget dependant...

Jiri Fiala · November 8th, 2009, 05:11 PM

How does Premiere fare with networked editing, anyway?

Harm Millaard · November 8th, 2009, 05:29 PM

Depends on who you are asking, but generally: lousy.

Adam Gold · November 8th, 2009, 09:09 PM

Adobe says not to even try it, but I'm not sure they were envisioning anything like this...

Steve Kalle · November 8th, 2009, 09:45 PM

Network editing=lousy. Well, sort of. If you use regular network drive mapping, then yes it is lousy. But if you use iSCSI, its far better. Network drive mapping shows the drive as \\Server\Video whereas iSCSI shows the drive as a normal E:\ (or whatever letter is assigned to it). iSCSI is designed for large workgroups with better reliability.

Stephen: I think I'm stating the obvious here when I say that you need centralized storage; so, its just a matter of how much you want to spend and how fast you need it to be.

If your editors need maximum speed, they can always download the video files to their workstation during that project. The Premiere and AE project files would be saved to the server so you could open it on your workstation to CC and whatever else rather easily.

I was just looking on B&H, and they have a good amount of fibre chanel equipment. FC is not as expensive as I thought.

How much storage space do you think you need excluding backup?

John Mitchell · November 11th, 2009, 08:22 PM

I would seriously look at Avid if your business keeps growing. No one does network editing better than Avid and their Unity product. Sure it is expensive but you save in the long run in productivity.

Unity works with other solutions as well (including Adobe and Apple) but clearly works best with Avid MC as you get true project sharing, even down to bin level (a bin that one editor has open gets locked for another.)

You could also look at third party Avid solutions mentioned in the thread, (EditShare and Facilis Terrablock) but I found them to be about the same cost for the actual network storage if a little cheaper per seat that Avid Unity.

The problem with other untried San solutions is dropped frames - only really important during capture and mastering, but most solutions built around a san will drop the occasional frame.

Stephen Armour · November 18th, 2009, 11:44 PM

I appreciate the comebacks!

As to editing in a networked environment, I never would have considered it with Premiere...ever...except the market is changing and so is Premiere. It appears the integration with CS5 will be even greater, and their 64bit changeover will certainly give them greater headroom.

Now, what remains to be seen, is if the 500 lb gorilla Adobe can actually do something to address this glaring weakness of the Premiere workflow. In the meantime, it seems we better suffer a few more months with our current cludge of x64s, multi-RAIDs, large file shuffling and too much disk redundancy, before jumping ship to something else. Any major moves now could cost a pretty penny.

The idea of a fast iSCSI SAN is quite appealing however, due to the fact it would then be our central file serve point and help to avoid some of the file duplication/confusion of multiple workstations all working on the same projects. All previews would still remain on each local machine, and we'd still have to break things into reels for editing, but it seems to work. The secret is getting Adobe to see that SAN as a "local disk".

At least our AE man won't go crazy on who has what and when, and all updated material will be in the same location.

Any "budget" questions for non-profits are usually met with a grim stare, but what I'm trying to gather here is an idea of what to shoot for. If I know the low, mid and high options, we can present those to interested parties in this venture.

As to storage needs, these are a long-term projects in multiple languages, each with a "master" and it's associated project files. The episodes aren't that long (25-28 minutes), but they often have quite a lot of VFX, a few stills, quite a number of text/titles, and a language dubbing track along with AFX and music. The English masters are the most complex and from them we generate the more flattened "masters" for the multilanguage projects. We use a mix of AE generated alpha layered text files and plain Premiere generated text files.

Sooooo.....to finally answer Steve's question: we have a moving target as we add languages. Since we need to have these at least near-online as our project output languages grow (21 now and maybe adding another 26 possibly in the near future). We need storage not only for master editing projects, but for generating additionally synced dubs and "burned-in text" sub-masters...all in HD. I would say 12 TB is easily where we'll be this time next year. We could possibly offline some of that, but who knows by then.

Makes me tired just typing it! Big HDD's are pretty cheap though, so aside from the hassle of having to plug in a drive to correct something and having to duplicate everything for security, we just limp along with CS3/4, Cineform, and lots of needs.

Steve Kalle · November 19th, 2009, 12:51 AM

When you start to look for the hardware, look into a rack case so you can use rack-mounted servers and storage. I suggest this because adding rack mounted storage down the road is very simple. As you might know, you can install Adobe on 2 computers so here is another idea: part of the rack idea, 1U rack servers using the new Intel 1156 socket with i7 cpu is very cheap. So, you could install a couple of these and use them for rendering so your editors can continue to work. I recently looked into a rack case and servers to build a small render farm for Cinema 4D and found I could get a 1U server for $500 not including hard drive, and this was with the Intel i5 750 or add another $120 for the Xeon i7 equivalent plus ECC ram(which is safer for long rendering). If you want the links to the hardware(all on newegg), let me know.

As to Premiere seeing the iSCSI drive, it would be just like seeing any drive installed in that pc. One of the main benefits of iSCSI is having the drive 'appear' as a local drive to the OS and programs.

Stephen Armour · November 19th, 2009, 11:15 AM

Quote:

Originally Posted by Steve Kalle

... If you want the links to the hardware(all on newegg), let me know.

As to Premiere seeing the iSCSI drive, it would be just like seeing any drive installed in that pc. One of the main benefits of iSCSI is having the drive 'appear' as a local drive to the OS and programs.

Souinds good Steve. Maybe if you post them here, someone else can benefit as well. Thanks. All the info I can get at this point is good. Appreciate it.

Steve Kalle · November 19th, 2009, 12:58 PM

Here is what I found: you can save $110 by getting the case and motherboard separately.

$370 - Supermicro Barebones Chassis w/Motherboard & PSU
Newegg.com - SUPERMICRO SYS-5016I-MR 1U Rackmount Barebone Server Intel 3400 LGA 1156 Intel Xeon X3400/L3400 series - Server Barebones

OR

$90 - Supermicro Chassis & PSU (Identical to Barebones but without motherboard)
Newegg.com - SUPERMICRO CSE-512L-260B Black 14" Mini 1U Rackmount Server Case w/ 260W Power Supply - Server Chassis

$170 - Same Supermicro motherboard as above Barebones
Newegg.com - SUPERMICRO MBD-X8SIL-O LGA 1156 Intel 3400 Micro ATX Intel Xeon X3400/L3400 series Server Motherboard - Server Motherboards

+
$240 - Intel X3440 Xeon 2.53GHz
Newegg.com - Intel Xeon X3440 Lynnfield 2.53GHz 8MB L3 Cache LGA 1156 95W Quad-Core Server Processor - Processors - Servers

$110 - Kingston DDR3-1333 4GB (2x2GB)
Newegg.com - Kingston 4GB (2 x 2GB) 240-Pin DDR3 SDRAM ECC Unbuffered DDR3 1333 Server Memory Model KVR1333D3E9SK2/4GI - Server Memory

So, $90+170+240+110=$610 All is needed is a hard drive.

For a regular motherboard($100) and Intel i7 CPU and non-ECC ram($70 4GB-2x2GB),
$90+100+290+70=$550

Intel i7 860 2.8GHz - $290 @ newegg or $220 @ microcenter

You still need a Rack case, which I found an open rack for under $200 and another few hundred $ for KVM switch and other accessories. And probably $150 for Windows 7 Pro x64 for each node.

Stephen, for your main server, you can get a 4U case with 16 HDD bays in the front.

Oh yeah, your AE guy might like the added render nodes so he could continue working on his workstation.

I would go for the server parts as they are generally built better and designed for heavy use.

Stephen Armour · November 19th, 2009, 02:24 PM

And controller card, drive, and OS suggestions? Just fishin...

Steve Kalle · November 19th, 2009, 11:18 PM

Quote:

Originally Posted by Stephen Armour

And controller card, drive, and OS suggestions? Just fishin...

Well, what I listed was just for render nodes. I would use something a bit bigger and more robust for the main file server, which could also be used as a render node depending on the OS.

For the File Server, you could use that Open-E OS which gives you iSCSI with regular NICs but no render node.

For an iSCSI hardware only solution, Newegg.com - Enhance Technology RS16 IP-4 Quad Intelligent iSCSI RAID Storage System - Server RAID Systems

It has 4 gigabit ports for iSCSI and 16 drive bays. What you can do is take those 4 gigabit ports and make them into pairs (ie team or bond) and set it up so you have 2 'drives' for each workstation to see. (ie one for masters and one for source)

With the Enhance Technology iSCSI, then you could get a render node or two.

I will refine what I suggested earlier:
Newegg.com - SUPERMICRO CSE-822T-400LPB Black 2U Rackmount Server Case w/ 400W Power Supply 1 External 5.25" Drive Bays - Server Chassis or
Newegg.com - SUPERMICRO CSE-743i-650B Black 4U Rackmount Server Chassis w/ 650W Power Supply 2 External 5.25" Drive Bays - Server Chassis

Both have larger power supplies, can take server size motherboards, have more drive bays and allow for more future upgradeability. Each Premiere/AE render node requires an OS - Windows 7 Pro x64 which is $140-150 for each pc. Having multiple server nodes requires more hardware like KVM switches and larger network switches. I assume you could get by with one dedicated render node so maybe spend a little extra and go with a dual-Xeon setup.

November 18th, 2009, 11:44 PM	#22
Stephen Armour Major Player Join Date: Dec 2005 Location: Natal, RN, Brasil Posts: 900	I appreciate the comebacks! As to editing in a networked environment, I never would have considered it with Premiere...ever...except the market is changing and so is Premiere. It appears the integration with CS5 will be even greater, and their 64bit changeover will certainly give them greater headroom. Now, what remains to be seen, is if the 500 lb gorilla Adobe can actually do something to address this glaring weakness of the Premiere workflow. In the meantime, it seems we better suffer a few more months with our current cludge of x64s, multi-RAIDs, large file shuffling and too much disk redundancy, before jumping ship to something else. Any major moves now could cost a pretty penny. The idea of a fast iSCSI SAN is quite appealing however, due to the fact it would then be our central file serve point and help to avoid some of the file duplication/confusion of multiple workstations all working on the same projects. All previews would still remain on each local machine, and we'd still have to break things into reels for editing, but it seems to work. The secret is getting Adobe to see that SAN as a "local disk". At least our AE man won't go crazy on who has what and when, and all updated material will be in the same location. Any "budget" questions for non-profits are usually met with a grim stare, but what I'm trying to gather here is an idea of what to shoot for. If I know the low, mid and high options, we can present those to interested parties in this venture. As to storage needs, these are a long-term projects in multiple languages, each with a "master" and it's associated project files. The episodes aren't that long (25-28 minutes), but they often have quite a lot of VFX, a few stills, quite a number of text/titles, and a language dubbing track along with AFX and music. The English masters are the most complex and from them we generate the more flattened "masters" for the multilanguage projects. We use a mix of AE generated alpha layered text files and plain Premiere generated text files. Sooooo.....to finally answer Steve's question: we have a moving target as we add languages. Since we need to have these at least near-online as our project output languages grow (21 now and maybe adding another 26 possibly in the near future). We need storage not only for master editing projects, but for generating additionally synced dubs and "burned-in text" sub-masters...all in HD. I would say 12 TB is easily where we'll be this time next year. We could possibly offline some of that, but who knows by then. Makes me tired just typing it! Big HDD's are pretty cheap though, so aside from the hassle of having to plug in a drive to correct something and having to duplicate everything for security, we just limp along with CS3/4, Cineform, and lots of needs. __________________ http://lightinaction.org "All in the view of the LION"

November 19th, 2009, 02:24 PM	#26
Stephen Armour Major Player Join Date: Dec 2005 Location: Natal, RN, Brasil Posts: 900	And controller card, drive, and OS suggestions? Just fishin... __________________ http://lightinaction.org "All in the view of the LION"

November 8th, 2009, 02:11 PM	#16
Harm Millaard Trustee Join Date: Aug 2006 Location: Rotterdam, Netherlands Posts: 1,832	Stephen, I just heard from my son. His comments: I can only give you good advise if I know the budget… In an ideal situation they would go for a Netapp storage solution with snapshotting at the block level for backup, so older files can be restored instantly. Such equipment starts at around € 20,000 and may run into the millions... Look here: http://www.netapp.com/us/products/st...ech-specs.html I can't really give an advise without knowing the budget available, but I do think that fiber channel is the best solution by far, but again, the equipment to use is largely budget dependant...

November 8th, 2009, 05:11 PM	#17
Jiri Fiala Major Player Join Date: Feb 2007 Location: Brno Czech Republic Posts: 453	How does Premiere fare with networked editing, anyway?

November 8th, 2009, 05:29 PM	#18
Harm Millaard Trustee Join Date: Aug 2006 Location: Rotterdam, Netherlands Posts: 1,832	Depends on who you are asking, but generally: lousy.

November 8th, 2009, 09:09 PM	#19
Adam Gold Inner Circle Join Date: Jan 2007 Location: Woodinville, WA USA Posts: 3,467	Adobe says not to even try it, but I'm not sure they were envisioning anything like this...

November 8th, 2009, 09:45 PM	#20
Steve Kalle Trustee Join Date: Aug 2009 Location: Chicago, IL Posts: 1,554	Network editing=lousy. Well, sort of. If you use regular network drive mapping, then yes it is lousy. But if you use iSCSI, its far better. Network drive mapping shows the drive as \\Server\Video whereas iSCSI shows the drive as a normal E:\ (or whatever letter is assigned to it). iSCSI is designed for large workgroups with better reliability. Stephen: I think I'm stating the obvious here when I say that you need centralized storage; so, its just a matter of how much you want to spend and how fast you need it to be. If your editors need maximum speed, they can always download the video files to their workstation during that project. The Premiere and AE project files would be saved to the server so you could open it on your workstation to CC and whatever else rather easily. I was just looking on B&H, and they have a good amount of fibre chanel equipment. FC is not as expensive as I thought. How much storage space do you think you need excluding backup?

November 11th, 2009, 08:22 PM	#21
John Mitchell Contributor Join Date: Jul 2005 Location: Sydney Australia Posts: 873	I would seriously look at Avid if your business keeps growing. No one does network editing better than Avid and their Unity product. Sure it is expensive but you save in the long run in productivity. Unity works with other solutions as well (including Adobe and Apple) but clearly works best with Avid MC as you get true project sharing, even down to bin level (a bin that one editor has open gets locked for another.) You could also look at third party Avid solutions mentioned in the thread, (EditShare and Facilis Terrablock) but I found them to be about the same cost for the actual network storage if a little cheaper per seat that Avid Unity. The problem with other untried San solutions is dropped frames - only really important during capture and mastering, but most solutions built around a san will drop the occasional frame.

November 19th, 2009, 12:51 AM	#23
Steve Kalle Trustee Join Date: Aug 2009 Location: Chicago, IL Posts: 1,554	When you start to look for the hardware, look into a rack case so you can use rack-mounted servers and storage. I suggest this because adding rack mounted storage down the road is very simple. As you might know, you can install Adobe on 2 computers so here is another idea: part of the rack idea, 1U rack servers using the new Intel 1156 socket with i7 cpu is very cheap. So, you could install a couple of these and use them for rendering so your editors can continue to work. I recently looked into a rack case and servers to build a small render farm for Cinema 4D and found I could get a 1U server for $500 not including hard drive, and this was with the Intel i5 750 or add another $120 for the Xeon i7 equivalent plus ECC ram(which is safer for long rendering). If you want the links to the hardware(all on newegg), let me know. As to Premiere seeing the iSCSI drive, it would be just like seeing any drive installed in that pc. One of the main benefits of iSCSI is having the drive 'appear' as a local drive to the OS and programs.

November 19th, 2009, 12:58 PM	#25
Steve Kalle Trustee Join Date: Aug 2009 Location: Chicago, IL Posts: 1,554	Here is what I found: you can save $110 by getting the case and motherboard separately. $370 - Supermicro Barebones Chassis w/Motherboard & PSU Newegg.com - SUPERMICRO SYS-5016I-MR 1U Rackmount Barebone Server Intel 3400 LGA 1156 Intel Xeon X3400/L3400 series - Server Barebones OR $90 - Supermicro Chassis & PSU (Identical to Barebones but without motherboard) Newegg.com - SUPERMICRO CSE-512L-260B Black 14" Mini 1U Rackmount Server Case w/ 260W Power Supply - Server Chassis $170 - Same Supermicro motherboard as above Barebones Newegg.com - SUPERMICRO MBD-X8SIL-O LGA 1156 Intel 3400 Micro ATX Intel Xeon X3400/L3400 series Server Motherboard - Server Motherboards + $240 - Intel X3440 Xeon 2.53GHz Newegg.com - Intel Xeon X3440 Lynnfield 2.53GHz 8MB L3 Cache LGA 1156 95W Quad-Core Server Processor - Processors - Servers $110 - Kingston DDR3-1333 4GB (2x2GB) Newegg.com - Kingston 4GB (2 x 2GB) 240-Pin DDR3 SDRAM ECC Unbuffered DDR3 1333 Server Memory Model KVR1333D3E9SK2/4GI - Server Memory So, $90+170+240+110=$610 All is needed is a hard drive. For a regular motherboard($100) and Intel i7 CPU and non-ECC ram($70 4GB-2x2GB), $90+100+290+70=$550 Intel i7 860 2.8GHz - $290 @ newegg or $220 @ microcenter You still need a Rack case, which I found an open rack for under $200 and another few hundred $ for KVM switch and other accessories. And probably $150 for Windows 7 Pro x64 for each node. Stephen, for your main server, you can get a 4U case with 16 HDD bays in the front. Oh yeah, your AE guy might like the added render nodes so he could continue working on his workstation. I would go for the server parts as they are generally built better and designed for heavy use.