View Full Version : Field and offload workflow


Craig Seeman
October 24th, 2007, 08:19 AM
I thought field and offload workflow warranted its own thread. I'd also include options not yet on the market but that we'd like to see.

I'll list a few and maybe others can share thoughts and list more ideas about cost, practicality, risk.

SxS to laptop and
- to internal laptop drive
- to external drive
- to internal Blu-ray drive

SxS direct to drive mechanism
- similar to Panasonic's P2 Store
- card to XDCAM record drive direct

and of course the possibility that two 16GB cards (100min@35mbps, 140min@25mbps) or two 32GB cards (200min@35mbps, 280@25mbps) may preclude the need to offload during a shoot at all. Now if the card prices dropped fast enough relative to the cost of the other workflows that would be idea. A bit of a fantasy maybe.

Alex Leith
October 24th, 2007, 12:40 PM
One suggestion was to use an iPod and either the camera directly or the SBACUS10 SxS Card Reader (using the iPod camera connector accessory)...

The success of this depends on the protocol used by the Sony SxS Card Reader and the transfer protocol and/or the way the EX1 stores files on the card.

http://docs.info.apple.com/article.html?artnum=301058

If I'm reading this correctly, the iPod will most likely support transfer of footage if the card reader uses "PTP", but most likely not if it uses "Mass Storage".

Malcolm Hamilton
October 24th, 2007, 01:16 PM
One suggestion was to use an iPod and either the camera directly or the SBACUS10 SxS Card Reader (using the iPod camera connector accessory)

Can anyone tell me how many minutes of EX1 footage you could hope to fit on the biggest iPod out there (160GB I think)?

Also (and I realize this is all hypothetical at this point, with everything depending on whether the card reader uses PTP), how would the camera, or card, connect to the iPod?

Thanks,
Malcolm

Craig Seeman
October 24th, 2007, 01:18 PM
I believe some HVX users were looking at the iPod route and found that they had to reformat the iPod (so it was, in effect, no longer usable as an iPod) and that the transfer rate was very slow. I guess this is why the P2 Store exists (and sells) otherwise many folks would go the iPod route.

I don't expect that to be any more successful with SxS.

One MIGHT hope for an affordable SxS to protable hard drive solution sans computer.

An Apple MacBookPro with Express34 port starts at about $2000.

My understanding also is that Sony's SxS to USB's xfer data rate is slower than going direct from card to laptop. 250Mb/s. And you'll need to power it.
This is if your laptop doesn't have Express34. (MacBook $1100 plus this $235 might work).
http://www.sonybiz.net/biz/view/ShowProduct.action?product=SBAC-US10&site=biz_en_GB&pageType=TechnicalSpecs&imageType=Main&category=XDCAMVTRs

There's the PDW-U1 which can be used as an XDCAM HD disc writer (probably around $2500 street) which can be hooked up to a laptop.

What I'd like to see is direct SxS to Hard drive or to disc burner with no computer "in between" though. Such might be "reasonable" if priced under $2000.

Malcolm Hamilton
October 24th, 2007, 01:32 PM
An Apple MacBookPro with Express34 port starts at about $2000.
//What I'd like to see is direct SxS to Hard drive or to disc burner with no computer "in between" though. Such might be "reasonable" if priced under $2000.

Hi Craig,
Wouldn't you be better off with a MacBook Pro, if it costs about the same as the direct-to-hard-drive/no-computer solution - - if such a thing is released, and costs the same as a MacBook Pro?
Malcolm

P.S. I am sorry that the iPod doesn't look like a good solution. So portable; plus, I've always wanted one but music has never been a good enough excuse...

Alex Leith
October 24th, 2007, 01:35 PM
I'm not sure how fast the "real-world" transfer rates for the newest iPod HDs are, but don't forget we're talking about less data per minute of footage than with DVCProHD. Still, I agree with you that an iPod is not going to be a quick solution.

According to Toshiba the transfer rate for the 160GB HD is 66Mbps (which would potentially give faster than real-time transfer for 35Mbps XDCAM HD)

As far as laptops are concerned - although USB has a slower maximum throughput than ExpressCard I suspect it's a moot point, as the laptop's HD will probably limit the actual throughput before either interface reaches saturation point.

Craig Seeman
October 24th, 2007, 01:50 PM
Malcolm, yes price is important from top to bottom. There's also workflow and portability. A laptop is nice because you can screen and even do rough edits to show a client once you dump a card, which should be very fast with the Express34 port in the computer.

On the other hand there may be shoots were I just want to pull a small "box" (battery/self powered hard drive with Express34 interface).

A lot of this depends on how card prices go and how quickly they drop.

Currently it seems 8GB - $500, 16GB - $900. My guess 32GB would be around $1600. Cut those prices in half and one may not need to offload at all except for very long shoots. Two 32GB cards would be 200 minutes (3hr 20min) at 35mbps and 280 minutes (4hr 40min) at 25mbps.

I can't see spending $3200 for two 32GB cards compared to $2000 for a laptop.

If it's low volume production of the cards that keep them high there is one issue to think about. Whereas Panasonic uses P2 accross most of its line, Sony has no other device that uses SxS. What's going to drive down the price of SxS?

So far Laptop (MacBookPro or equivalent Sony Vaio) is the most cost effective solution but it doesn't make sense to me that the market won't offer price competitive job specific alternatives.

Christopher Barry
October 24th, 2007, 05:13 PM
SxS to laptop and
- to internal laptop drive
- to external drive
- to internal Blu-ray drive


If initial cards are 8GB, then instead of buring "to internal Blu-ray drive", perhaps burn each 8gig card to a dual-layer DVD, which has up to 8.5GB of available space. Any views on this, especially since I can use my existing laptop?

Also, should the longevity of different media (DVD, HD-DVD, B-R) be of concern?

Malcolm Hamilton
October 24th, 2007, 05:15 PM
I agree, Craig. I think the market will produce a few options before long... It'll be interesting to see who does what, and when.
Cheers,
Malcolm

Piotr Wozniacki
October 24th, 2007, 05:31 PM
Also, should the longevity of different media (DVD, HD-DVD, B-R) be of concern?

I would trust any of these formats for archiving. Pro disk, yes - but not consumer format.

Therefore, field off-loading to laptop - yes, but to HDD.

Christopher Barry
October 24th, 2007, 07:17 PM
Yes, in the field, I would off-load to two separate HDD, and at some point (in the field or end of day) archive to DVD dual-layer (if 8GB cards), pending the next wave of cost effective mass storage and media archiving, such as HD-DVD, B-R or other.

Alexander Ibrahim
October 24th, 2007, 10:47 PM
As far as laptops are concerned - although USB has a slower maximum throughput than ExpressCard I suspect it's a moot point, as the laptop's HD will probably limit the actual throughput before either interface reaches saturation point.

That depends on the laptop HD itself, and it assumes that you are outputting to the laptop HD.

I for one plan to copy from the Expresscard slot to a firewire drive. That arrangement should allow about 1GByte per minute transfer.

This gets interesting when I have to go HD/SDI to an ioHD through the mac to an eSATA drive for higher quality capture in ProRes.

If I have to mix those uses on set then either I have to swap things around quickly or if things are bus enough I may have to use two MacBooks.

Alexander Ibrahim
October 24th, 2007, 11:10 PM
If it's low volume production of the cards that keep them high there is one issue to think about. Whereas Panasonic uses P2 accross most of its line, Sony has no other device that uses SxS. What's going to drive down the price of SxS?

Well from what I can tell, what is going to drive down the price of these cards is their general usefulness as laptop storage expansion.

Think about it- these are fast flash hard drives. They are going to use relatively little power, and will work with a lot of modern laptops over a fairly fast bus- quite possibly outperforming the installed SATA drive.

For what its worth, I also think we'll see more SxS based cameras from Sony in the XDCAM range. It will probably take them as long as it took Panasonic to transition- but remember that Panasonic still has tape based DVCPRO HD cameras for sale. (The Varicam at least, there may be others.) Sony brags about the value of the mechanical engineering in their Betacam derived tape transports- like HDCAM and HDCAM SR. I don't think they'll go away unless they can find some other equally valuable advantage in the solid state age- or until competition forces their hand.

Alexander Ibrahim
October 24th, 2007, 11:41 PM
I've seen some reservations about archiving to DVD or other consumer media.

In the end I want to express that archiving on anything other than film is unproven.

Some have talked about how unreliable the mechanical parts of HDDs are.

Others have pointed out that archiving to DVD is dangerous, due to the "bit rot" they've experienced.

The fact is that if you have data archived on digital media, then you must periodically verify that media. You must also transfer that material to newer digital formats before the current format is archaic and unreadable.

(Not merely obsolescent in the sense of being outdated, but rather in the sense of being unable to find playback devices or systems that understand the data.)

In that sense DVD and CD have been the most reliable archival formats for digital media.

Like all media, you must use archival quality media and storage techniques in order to preserve the media for any length of time. I have archival CD's in basic jewel cases more than a decade old with full readability.

Mostly this amounts simply to keeping the media in a dark, dry place with relatively cool temperatures. Disc media should be stored in separate containers. A good container will support the disc both by the spindle hole and the edge to prevent distortion of the disc shape. It will also keep the disc's data bearing sides from contacting a surface.

"Pressed" ROM media will last much longer than WORM or RW media.

Here are two links for more information on digital archival in general.

http://memory.loc.gov/ammem/techdocs/digform/DigForm_Intro_v04.pdf

http://www.digitalpreservation.gov/formats/

If you follow the precautions I go over here then DVD media should be suitable archival material for most of the work we'll do here.

One additional strategy is to archive on multiple formats. For example on hard disk and DVD. This buys you some insurance against a format becoming archaic- because its extremely unlikely that two separate formats will become archaic at once with such rapidity that you can't transfer your media before the other one becomes archaic.

Alex Leith
October 25th, 2007, 01:24 AM
That depends on the laptop HD itself, and it assumes that you are outputting to the laptop HD.

I for one plan to copy from the Expresscard slot to a firewire drive. That arrangement should allow about 1GByte per minute transfer...

Don't forget that the "shock resistance" capacity of 2.5in HDs is much greater than 3.5in HDs. Personally I'd much rather be using the smaller slower drives in the field, as they are likely to get knocked around a bit.

Paul Ramsbottom
October 25th, 2007, 01:48 AM
I've seen some reservations about archiving to DVD or other consumer media.

In the end I want to express that archiving on anything other than film is unproven.

Some have talked about how unreliable the mechanical parts of HDDs are.

Others have pointed out that archiving to DVD is dangerous, due to the "bit rot" they've experienced.

In that sense DVD and CD have been the most reliable archival formats for digital media.

Like all media, you must use archival quality media and storage techniques in order to preserve the media for any length of time. I have archival CD's in basic jewel cases more than a decade old with full readability.

"Pressed" ROM media will last much longer than WORM or RW media.

One additional strategy is to archive on multiple formats. For example on hard disk and DVD. This buys you some insurance against a format becoming archaic- because its extremely unlikely that two separate formats will become archaic at once with such rapidity that you can't transfer your media before the other one becomes archaic.

Heh, I still have my original Joshua Tree CD, bought on day-one in 1987 - has it really been 20 years!!!!

I can remember at the time that everyone had their knickers in a right old twist about how CD's would degrade and be unplayable in a few years.

It's funny, I like having it in my CD rack but it's really not that important to me.

Piotr Wozniacki
October 25th, 2007, 01:53 AM
While this is true about pressed ROM disks, I'm afraid what we burn is not reliable at all...

Craig Seeman
October 25th, 2007, 07:40 AM
It may depend on what one means about "archive."

There's "I'm shooting a feature/documentary and I want it to last forever." And there's "I'm shooting a corporate video/tv spot and I want it to last for five years so I can do revisions or use the clips in future projects."

If it's the later, DVD should be safe as long as you buy quality discs.

If it's the former, you'll probably want to move it to another format every 5 to 10 years otherwise do a film xfer (!!!!).

BTW having a laptop record to dual layer DVD also resolves the "what do I hand the client after the shoot" issue. I actually think this work flow is BETTER than HDV. There's so many deck/camera HDV incompatibles because of the variation in the formats. Sony's free viewer utility will mean anyone with a reasonable computer will be able to view the material you shoot, no deck needed. With ingest/burn speed the turnaround after a shoot should be reasonably short for all but the longest (all day) shoots. And, unlike handing tapes to the client, you have your copy on hard drive in case something goes awry with the burned discs.

Alexander Ibrahim
October 25th, 2007, 10:45 PM
While this is true about pressed ROM disks, I'm afraid what we burn is not reliable at all...

For the record, that disk I was talking about is from July 30 1993, it has a backup copy of the Windows NT Advanced Server 3.1 disks on it.

This disk I used is actually marked CD-WO, and is probably quite a bit older than 1993. (I don't remember when they changed the name to CD-R, but I thought it was 1990 or so.)

Its a phthalocyanine disc with a gold reflective layer. Sadly I have no NTAS 3.1 install key- and probably no hardware that can run it- so I'll probably chuck it.

I think that qualifies as a disk I burned.

Paul Ramsbottom
October 26th, 2007, 03:03 AM
My [U2 CD] point was more about the value of all this stuff, if you're not in the the big-league and being submiited to the Library of Congress, then fuggetaboutit - no-one's gonna care about your pay-the-rent corporate vids, home movies and indie film projects - peace :)

Alexander Ibrahim
October 26th, 2007, 09:15 AM
My [U2 CD] point was more about the value of all this stuff, if you're not in the the big-league and being submiited to the Library of Congress, then fuggetaboutit - no-one's gonna care about your pay-the-rent corporate vids, home movies and indie film projects - peace :)

Oh I agree OH so much.

I have some DV footage of a USFSA regional championship and really nobody cares at all. The show I was shooting for wasn't picked up, and the event is so not current no one, except the skaters at the event, might be interested.

I have some footage of the Pentagon from 9/13, also DV, and again there's barely a reason to archive that. In fact going forward for me, just about any footage I'd want from those days will have been acquired on 35mm back then, not DV. I keep it because the DV footage is free to me at least, & free might matter.

At the same time though, if you want some footage archived then you better have some sensible archival plans for it. Plans you can implement. Unlike the big guys, the Library of Congress won't be doing my archival anytime soon.

For most of us that means the copy to newer media every so often type plans.

For example that Pentagon footage is preserved on HDD and DVD as DVCPRO 50 footage. Of course I also still have the tape, but at this point I know it has some dropouts... so I may use footage from the HDD as my "master" for future re-use of the footage.

Paul Ramsbottom
October 26th, 2007, 10:52 AM
And obviously Alexander, that comment wasn't aimed just at your stuff :)

Alexander Ibrahim
October 26th, 2007, 12:42 PM
And obviously Alexander, that comment wasn't aimed just at your stuff :)

Oh, I know. I don't mean to seem remotely offended, because I'm not.

I just wanted to talk more about your point, which I really agree is valid. You mentioned the U2 CD, I wanted to give a more direct example from my work in support of what you said.

Maybe you thought I was being tongue in cheek, but I wasn't.

As important as that stuff seems at first, the truth is that it isn't that interesting or useful six or seven years after the fact. Doubly so given the acquisition format.