Studio storage / networking advice please~

[Julian Ott]

This is my first post here, and I welcome any advice you have. I work in a multimedia artist's studio and we seem on the verge of an upgrade. Essentially what we need is a solution for video storage, (backup and editing), and networking. You may see this as two separate solutions, but I'm including both as there is overlap.

Our current scenario: My boss uses a Mac Pro for 4k editing and has several SSD and larger drives within the machine itself. Each of his 3 assistants has a PC or Mac machine in addition for related media work. We often have to access files in each others' computers during our workflow as projects have become more involved, cross-platform, and using multiple media sources (such as 3D, still images, etc.). 4K editing really only needs to happen on his Mac Pro machine. Right now I have a d-link router handling all the traffic and enabled sharing between all the macs and PCs, but it has issues. We have about 15 TB of data we want frequent and easy access to.

Problems:

- Streaming video from his machine, which we need to do sometimes, is nearly impossible over the network.
- Sometimes one PC or Mac cannot recognize another PC or Mac's network drive, but other computers can. There doesn't seem to be any rule for when or why this happens but a restart usually works.
- Transferring files between machines is incredibly slow at times, especially between Macs and PCs. We use USB thumb drives sometimes––it's ridiculous.
- We end up having to open up machines and switch out drives in order to have access to all the files we need because of limited space on drives / in computers.

Aspirations:

- A storage solution of at least 15 TB fast enough to stream encoded 4k .mov files to 4 computers (but only 1 at at a time).
- A fast storage system with at least a terabyte of space for 4K editing (can be internal to the main editing Mac Pro)
- Networking solution so that Macs and PCs play together nicely.
- Backup system that selects folders over the network from various computers. Can run overnight.

Possibilities:

- We can upgrade to an external thunderbolt / other interface HD solution
- We will upgrade the Mac Pro (currently a bit slow for 4k editing)
- We will replace the d-link router

Caveats:

- We can't go all-mac because of certain applications--there must be at least one PC.

Questions:

- Would Apple's Airport be an ideal networking solution?
- Would a central data storage solution be best, or would networked local storage make more sense for most frequently accessed files?

I'm both a Mac and PC user, and have never had a multimedia system issue that a simple router, a few SSDs, ghost / time machine external backups couldn't solve. The kind of system we want seems a little over my head. If you have advice or could point me to where I can learn what to look for I would be sincerely grateful. We're really quite flexible on solutions since we can upgrade a lot of what we have.

[Tim Lewis]

Hi Julian

Welcome to the forum.

I will just give you a bit of information on the IT side of things as this is where I have the most experience.

I would suggest that you ensure you are running a router that has Gigabit Ethernet ports and that you network via Cat 5e or Cat 6 to ensure the best throughput possible. You will also want to ensure that all computers and other wired devices that connect to the network are using Gigabit NIC's (network interface cards). It may also be better to assign fixed IP addresses to each device and ensure that they are in the DHCP pool range. This will avoid potential network issues with IP address conflicts.

A router with a DHCP range of, for instance, 192.168.0.1 - 192.168.0.254 will have 254 allowable IP addresses of which one is allocated to the router as the DHCP server and gateway. This would usually be the top of the range, i.e 192.168.0.254. All other devices can be assigned an IP address in the range. Having fixed IP addresses in a specific part of that range, such as 192.168.0.50 - 100 will keep them out of the way of leases being assigned by the DHCP server, which usually starts at the bottom of the range. This is where wireless devices such as phones and tablets will probably get their IP addresses assigned.

Once you have a stable network configuration like this, you can basically forget about network issues and concentrate on other factors, which I will leave to those with more experience in those areas.

HTH

[Julian Ott]

Tim,

Thanks so much for your advice. I was wondering about IP conflicts, since we have lots of wireless (phones, laptops) devices attached too. I'll definitely work on that on Monday.

Cards are all gigabit but cables are just bits we've gotten from here or there so I'll buy some new ones next week as well.

Thanks again. I'll let you know how that part of it goes.


julian

[Seth Bloombaum]

Originally Posted by Julian Ott

...Cards are all gigabit but cables are just bits we've gotten from here or there so I'll buy some new ones next week as well...

Have you confirmed that the router is gigabit at all ethernet ports as well?

I've only been involved in this sort of project from the specification side, I guess I mean as a user.

One rule of thumb I've come to value: Take your expected storage needs and double them! Maybe you've already accounted for that in your 15TB estimate.

When you say "stream a 4K file" from one PC to another, are you asking to support a real-time stream? Why? Would user B actually play/edit a file form user A's machine?

And what kind of 4K video file are we talking here? I can't spec connectivity based on what you answer, but the people who can spec will want to know just what the bitrate of that file is if you're looking for real-time streaming of it.

[Julian Ott]

Hi guys,

Thanks for the networking tips. I have confirmed everything is gigabit (router & cards) on each machine. I will play with addressing a bit later in the week when I have time to learn that.

Seth: Yes, 15 TB is 6 more than we currently use and that has accumulated over some years so I imagine 15 should work over the next 3-4 years.

As far as streaming goes, we would never do editing on another machine without first bringing over the files, and 4k editing is out of the question anyway on anything but the Mac Pro. We do however need to stream files as a preview to check them. We often have to send clips for press, curators and other reasons and to do that well we need to be able to see what version of what something is. I know it sounds obscure but it happens to be something we do often. Most of the files are lower than 4k but some are like 3840 x 1080 or something like that.

edit: and the files are variable but usually prores 422 HQ and sizes such as 3820 x 1080

I assume ethernet wouldn't be fast enough to do that. Am I wrong?

[Bruce Dempsey]

Maybe check out wd mycloud

[Julian Ott]

Bruce: I just checked out the reviews for MyCloud, but it seems far to slow for anything except backing up. Its largest con seems to be performance.

So I realize I'm really asking for three separate solutions. Apple Airport seems like a good solution for our networking issues, something like Mycloud could work for backups, and a fast thunderbolt / pcie box could work for editing. Are three separate purchases the best solution, or are there any other options?

Thank you!


julian

[Seth Bloombaum]

MediaInfo is a great pc/mac freeware that will allow you to see the specs of any video. (Select an advanced view like "Tree" for full specs.)

Won't help with content versioning of course, but as far as figuring out if you're looking at the 4k or a 2k, it would be great for discovering clip specs.

Then... getting into disciplined use of descriptive file names like ATTMarketingFeb2014YoutubeVersion2.mp4 is very helpful for everyone - including the editor! Maybe you do that already.

Obviously you'll still need to do some previewing, but these things do help!

Originally Posted by Julian Ott

...edit: and the files are variable but usually prores 422 HQ and sizes such as 3820 x 1080

I assume ethernet wouldn't be fast enough to do that. Am I wrong?

Not working in 4K at 422... I'm sure that info is out there. Fiber Channel networking?

[Jim Michael]

I wouldn't trust a wireless connection to a mission critical high throughput requirement. I'd start looking at network storage (e.g. Synology DiskStation) and analyze the time horizon required for archive access. If more than a couple of years look at an LTO tape drive for archiving data. Later as your storage needs grow you can migrate to a full fledged SAN with full redundancy.

[Jim Andrada]

I think you somehow need to find out where the bottlenecks are. There are 10Gb Ethernet cards and switches available (but quite a bit more expensive than 1Gb) IF network bandwidth is your problem, which it may not be. Do you know how your network is configured re block sizes - IIRC at one time there was a handshake across Ethernet for every 1500 bytes transfered which really slowed down large transfers - I know there are ways to increase this, but standard IP protocols are not optimized for large file transfers. The high performance networking systems I'm familiar with have all implemented proprietary protocols across an Ethernet network and often require more expensive low-latency switches as well.

However, there are a lot of other places to get bogged down - the computers themselves, the Network Interface Cards, the software, the HDD's, Drivers - the list is long.

And I'm probably of no use in figuring out exactly where your problem lies, other than to point out that I know of proprietary systems that can transfer data at about 80% of the maximum rated speed of a network, but they all use proprietary protocols and low-latency switches to make it work. One way to think about it might be to calculate the actual bit rate you're trying to transmit and see what % of 1Gbit/sec it is. I doubt you'd get more than 30 to 40 net MEGABYTES per second out of a normal network, but I could be off quite a lot - I'm not in the tech end of things anymore (which might be obvious!) but I am involved with a couple of high data rate over Ethernet products and I know the kind of things the engineers are always complaining about.

Anyhow, I'd start with figuring out the net data transfer rate you want to get out of the network and then find someone smarter than me to help you configure accordingly in order to meet it.

[Tim Lewis]

Julian,

Jim is right that the traffic across an IP network is divided into 1500 byte packets. Much like the light that enters your camera the data stream is not a continuous flow. Each of these packets also contains a header that tells the networking equipment where the packet is to go and under what protocol the packet is being sent. Network equipment is designed to handle this at the throughputs appropriate to the connection interface.

There are protocols that allow packets to have up to around 6000 bytes, but "jumbo frames" are NOT supported by all networking equipment and could potentially cause more trouble than they are worth.

Then there are the MTU settings on each computer, which defines the maximum packet size the computer will send, the incorrect setting of these can cause weird behaviour, like web pages not loading passed the header.

As Jim Says 10Gb ethernet is available but expensive and you would have to ensure all devices were capable this speed if they were on the critical data path. You could move the DHCP server/ router from a central hub role to a end point as it doesn't need to serve that information at high speed.

Then there is network topology, will a star or ring layout work better? Star may be higher speed, ring will be more resilient. You have lots to think about.

[Julian Ott]

Thanks for the notes, guys. I've made progress. Before considering 10 GB ethernet cards that cost more money or changing packet size (which is a bit beyond my level of knowledge), I'll work on the network.

Ethernet:

I read up on how to set up static IPS, but remembered I hadn't updated the dd-wrt firmware on my d-link router in a couple years, so I updated the firmware and file sharing between the macs is working much better. At the same time, I switched our CAT-5 cables out with new ones.

I'm able to stream at much more satisfying speeds. Now, if I hit spacebar on my macbookpro for a 1080 H264 file on the macpro, it works fairly quickly. That's very suitable for me. In addition, moving large video files over to my machine such as to take still shots works great. It might have been the network cables...

Remaining issue:

- I can access the PC from any Mac, but not any single Mac from the PC, but I'll keep working on that. I'll try starting over with the mac-pc sharing in the next few days.

Storage:

My boss will get a new Mac Pro later this year and we will
1) use an external thunderbolt storage device for editing on the MacPro
2) use a secondary external solution for storage and backup, and backup using timemachine on the mac pro.

Mediainfo is a great piece of software.

Tim, Jim, Jim, Seth, and Bruce thanks for all the suggestions. I realized that the three issues needed to be handled separately. Even for your recommendations that I can't implement, it gave me a scope for something I'm learning more about. Video really makes you learn to tweak a system! Now to move on to finding bottlenecks with Black Magic and mediainfo.

peace!

[Chris Medico]

We have a network setup for our media and we can easily serve several streams of 220mb material from the server over our gigabit network. This obviously isn't 4k. What we had to do to make it work is install a managed switch and have one ethernet port per user on the server itself. As example if you have 4 edit systems then you will need 4 ethernet ports in the server. Each edit system should be "bound" to its corresponding ethernet port on the server by the software in the switch.

On the server side you will need to run a fast RAID setup with a dedicated RAID controller card. I would recommend a minimum of 5 drives in the RAID.

Even with a setup like this I suspect you won't be able to do 4k editing with the media residing on the server. It would be best to have the master media on the server and create 1080 proxy files for the online edit the conform to the master media when you are done. That workflow will work well with a setup as described above.

[Julian Ott]

I was considering the Netgear R7000 router because it has a USB 2 and 3 port for backup options, has NAS functions, works with time machine, printer sharing, and performs well. However, it looks like we're going to have 6 machines on the network, so typical 4-port ethernet routers won't work and I'm thinking about a switch in combination with a new router.

Chris, may I ask what kind of managed switch you installed? Is it just to connect the ethernet ports on the server to each machine...and...is this hard to do within the software? Is this to overcome the bottleneck of a single gigabit ethernet port?

Also, is it difficult to install a switch under a router just for the media server and connected machines?

We won't be editing 4k over this network, but being able to stream it would be very ideal.


Thanks.

[Chris Medico]

We bought a really nice Cisco switch - used - for pennies on the dollar off Craigslist. If you look around you'll see we are awash in great networking gear just sitting around. You don't have to buy new unless you want to spend more $$. Its one of the 3000 series Catalyst switches, sorry I don't have the exact model here.

Having a managed switch is nice because you can configure it to dedicate its backplane bandwidth across the ports you will be doing the video on. It reduces the latency and any of the brief stutters of data that can happen when you throw a lot of data all of a sudden at an unmanaged switch.

Configuration isn't so hard once you read up on how to do it. If that just isn't your bag I'm sure you can have a local computer person write you a config file to upload to the switch.

Installing the switch is a piece of cake. You run a cable from one of the ports on your router to a port on the switch and plug all the computers and servers into the switch. Your network WILL thank you for doing that. Even if its not a managed switch.

Yes, we made this choice to reduce the latency on the ports and to make sure we had a full bandwidth connection between each edit station and its corresponding port on the server.

What we have isn't fancy and for the most part it is stuff we hacked together. What we gained is being able to edit on one common pool of media and share timelines between us instantly. It has made a HUGE difference in productivity.