Capture uncompressed
 

Is it possible to capture FireStore Raw Data to an uncompessed NLE Quick Time environment? - In the same sense that we capture "uncompressed" DV into 10 bit for heavy effects work. If so do we still need a special configuration of Video cards and fast arrays? Transfer rate?
Thank's for any info on this topic.
Bill Anderson.

I'm not following your question. DV is not 10 bit nor uncompressed.
A firestore will just store what it gets from the camera over
firewire which is plain DV.

You cannot go any better with any DV cam since it only has DV
out (and analog).

Rob it's not that uncommon to capture DV to an "uncompressed' 10 bit environment in order to sustain as little degradation as possible with heavy effects and renderings, and subsequently output to film or i.e digibeta etc.
I can't quite figure out if that option is still open with the firestore drive. Can it deliver the larger bit rate for so called uncompressed environments etc? I think I might be making the mistake of veiwing the drive as a tape deck rather than a holding device which is simply accessed by the nle.
Any thoughts?

Well you might capture to uncompressed 10 bit but that still does
not give you anymore starting bitdepth since DV is 8 bit. So unless
you have a camera with uncompressed SDI out I don't see how
you can gain anything. Ofcourse, once it is in that format you have
more room to play in, but that can also be done after a DV
capture so that doesn't change a thing.

Again, a DTE device simply records the signal from the camera
which is DV in this case. Yes it is also a normal drive so if you
hook it up to a computer you can put whatever you want on the
disc. The DTE will however not know what to do with anything
it doesn't understand.

So yes you can store an uncompressed 10 bit file format on that
drive if you wanted to. But it I don't see what this will gain you
since you can use any drive you want to do that.

Rob you can not "use any drive" to capture or work in an 10 bit environment. Most stock drives are simply not capable of handling the added bit fate, which is considerably more than DV firewires 3.6 MB/sec. "Faster" or array drives are usually required for "uncompressed" work. And I am still not sure whether this hard drive will afford the possibility of working in an equivalent manner. Again, I might be missing something obvious here, but whatever that is it is not the benefit of working in 10 bit even though DV is 8 bit.

I have to disagree there Bill. I can do uncompressed 10 bit with
a harddisk that does 1 MB/s if I wanted to. It just wouldn't be
realtime. Now that's not something you have said.

Uncompressed 10 bit will take approx. 37 MB/s. So if you want
to do it realtime then you will either have to use a disk array
indeed or go with a lossless compression codec.

All I said was you can STORE the information on such a drive
if you wanted to. I've never said it would be fast enough or
large enough (37 MB/s fills up a drive FAST).

Still not sure why you would want to do this and looking from
your last line in your previous post neither do you Bill?

But back to the original question. I've re-read it and perhaps
have a better idea at what you are doing or wanting to do.

It sounds like you have a camera that you hook up to some
uncompressed 10-bit system via I presume an analog connection
(could also be firewire, but those systems rarely use it).

Then you want to know if you can also do this with a DTE device?

Please try to describe the workflow as detailed as possible please.

Rob, you seem to miss the point of working in an "uncompressed" environment. But why I, or anyone else, would choose to do this is moot. My query was aimed at whether this device would allow for this workflow and not "why". There are many valid reasons to decide on an "uncompressed 10 bit" workflow and you should understand this. Does this device's capture rate or any other factor preclude this being a viable option?

I can bring DV tape aquisition to my NLE at 10 bit uncompressed. Can I perform this with this hard drive? When I talk about an 10 bit DV environment to anyone with the most rudamentary knowledge of video I can safely assume that they understand the ramifications. And safely (usually) assume that they know that this involves more than just a 10 bit process, or for that matter a 16 bit process that we commonly use in Photoshop etc.

Can you tell me of a 1 MB/sec hard drive capable of capturing 10 bit "uncompressed video? - I am all ears.
I admit that there might be something obvious that I do not understand with this method of capture as it might (or might not) translate to a
10 bit 4:2:2 environment and that is why I have aked this reasonably simple question. Reading between the lines my guess would be that I continue to use tape for any "upconversions". ???

The drive can only capture what the camera puts out. All DV cameras output compressed 8-bit DV on the firewire. So that's what the drive will capture.

No camera (with the possible exception of the SDX900) outputs anything other than compressed 8-bit DV over the firewire.

Bill: I don't know what else I can say. Either one of us or the
both of us do not understand what the other is talking about.

I'm not so interested in the why either. I'm interested in trying
to understand where you are coming from and what you are
trying to do. It is often the case that people ask the "wrong"
question (not saying you do!).

In the end it's just a harddisk with a system around it, but from
a camera it can only capture 8-bit DV as Barry and myself have
pointed out. Can you transform this to 10-bit uncompressed,
sure. No problem. Can you store this back on the device from
a computer. Yep, no problem as well. Will it be fast enough /
realtime (this might be your "does it work" line). No.

I understand very well what the advantages of 10-bit are. I
also understand very well that most people do not like to work
in uncompressed for various reasons (harddisk speed and
storage requirements). I also know that a lot of applications
cannot work in more than 8 bits unless you go to professional
systems.

Again I was (and still am) trying to understand the exact
question and what the workflow is. Perhaps I'm just not
understanding it. But from the lack of other people responding
in this thread (besides Barry above) I don't know.

Thanks Barry and Rob for your input. I do understand that this is a drive like most others and that it will capture what the camera delivers: 8 bit DV
4:1:1. NTSC. But then so does tape, and yet it can be captured to a 10 bit uncompressed environment for, hell, whatever reason.

Rob, regarding the disc drive in question, you state, "Can you transform this to 10-bit uncompressed, sure. No problem."

Now we might be getting somewhere. How WOULD YOU transform this to "10 bit uncompressed" ? For output to, for arguments sake, Digi Beta.
Thanks again.

With the right sofware. For example, if you have FCP HD you can
simply load the footage in there and export to an uncompressed
10-bit format (as I understand it, I don't have FCP myself).

I don't own any software myself that can do this (as I said in
my previous post most software still works in 8-bit!). I can easily
export to 8-bit uncompressed for example, that's no problem.

When you say: "and yet it can be captured to a 10 bit uncompressed environment"

If I write some software I could even capture in 16 bit uncompressed
if I wanted to. You can *ALWAYS* expand footage to uncompressed
or to more bits whatever the source is (with the right tools).

The question is why would you want to and why do you need it.
Again, there isn't a lot of software in the pro-sumer market that
supports that. Only higher end gear.

It sounds like you have worked, or know someone who does,
with a system that allows you to capture in 10-bit uncompressed.
This is probably a professional system as well. And my guess is
that it is also analog only (or perhaps SDI). It's hard to compare
such systems because the DTE devices we are talking about are
native DV. So you can't hook them up to such a capture board.

But again, with the right tools (which such a system probably
has, not 100% sure on that) you can convert the file in the
digital world to 10-bit uncompressed.

Thanks Rob. Yes indeed I have access to a FCP system which, unlike my own, can capture to an 10 bit uncompressed environment through the usual means of fast array drives, video card etc. My reasoning (if you can call it that) was that there might just be a way, with the FireStore drive, to bypass the need for arrays and video card etc. and yet work in such a desirable environment. Desirable? Well, if you plan on doing a lot of effects, collor correction and chroma keying, it certainly helps to work in an 10 bit 4:2:2 color space. And it mostly has to do with the destructive nature of rendering, which is far less pronounced in this space. And when you're out to film, this is certainly not a bad thing.
Again, thanks for hanging in there.
I had this idea stuck in my head and couldn't quite resolve whether there was a way to cheat the footage into this space without the usual hardware.

Quote: "I can bring DV tape aquisition to my NLE at 10 bit uncompressed."

Bill,

Just out of interest, How do you ingest from tape to your uncompressed system?
That is, how is the tape deck attached to the editing machine for capture?

Mark

Mark there are various video cards on the market that, once installed into your computer, will allow you to capture SDI from your deck, (if it has SDI) with a BNC type video cable. The data rate is increased to the point where you will need a way to sustain capture speeds, which in turn means faster drives than are generaly found in off the shelf computers. The drives are stacked in various configurations, "arrays". It's not an inexpensive setup and "uncompressed" is not always required. Check out Marco Solorio's article
"DEPRESSED ABOUT UNCOMPRESSED?" @ www.kenstone.net Final Cut Pro site.
Hope this helps.

<<<-- Originally posted by Bill Anderson : Mark there are various video cards on the market that, once installed into your computer, will allow you to capture SDI from your deck, (if it has SDI) with a BNC type video cable. The data rate is increased to the point where you will need a way to sustain capture speeds, which in turn means faster drives than are generaly found in off the shelf computers. The drives are stacked in various configurations, "arrays". It's not an inexpensive setup and "uncompressed" is not always required. Check out Marco Solorio's article
"DEPRESSED ABOUT UNCOMPRESSED?" @ www.kenstone.net Final Cut Pro site.
Hope this helps. -->>>

I think I understand what you're asking, Bill. And, unfortunately, I think the answer is no - at least, not without some major modifications to the Firestore.

As I understand them to work (keeping in mind that I've only used one once, at a show) the video is already 'captured' as files on the Firestore's harddrive. That means, I think, that you're not going to be able to bring it directly into a different space than native DV. You will almost certainly have to bring the DV files into FCP and use it to convert them to uncompressed 10-bit (or use some similar workaround).

The only way I can see that you would be able to go directly to uncompressed 10-bit in FCP from the Firestore would be if the Firestore were actually capturing in uncompressed 10-bit, which I'm pretty sure isn't possible right now.

You might try calling the manufacturer and asking them, however.

Thanks Joshua. That makes sense. I think I was being a bit overly optimistic. However, the FireStore drive is an excellent looking tool. And who knows, perhaps a FireStore "uncompressed" capture disc (and HD) will be in the future - that's optimism.
Cheers.

Most likely. Isn't the new JVS HDV ENG camera going to be able to record uncompressed to disk if one has the necessary accessories? I may be wrong as I can't remember where I heard that.

I don't think you're going to see Firestore capture uncompressed. First, a single drive isn't fast enough. Second, there's no advantage.

The "correct" place to convert DV from compressed to uncompressed is inside the NLE, at the time you need it converted. You can do this in FCP by dropping DV files onto an uncompressed timeline. This won't improve the DV - you need to upsample or reconstruct the chroma for that - see my website www.nattress.com for my experiments in that!

The way FCP works, you could just edit entirely in DV for speed, and then just change the final timeline to uncompressed and render. If you need to take any shots into AE or Shake or whatever, you can export them to an uncompressed codec at that point, although, if you don't need to keep an alpha channel, use PhotoJPEG @ 100% quality as it's imperceptably worse than Animation 100% (apart from the lack of alpha) and will work better over multiple generations than any "uncompressed' 4:2:2 based codec.

If you were to bump DV to uncompressed in an external hard drive recorder, the only thing you'd get would be to run out of hard disk space faster.

Graeme

I must disagree with Graeme. You gain a lot by recording the uncompressed signal. Most significantly you retain all of your color information and greatly improve the dynamic range of your signal. This is a very big bonus for those using DV for features and shorts.

Converting to uncompressed in a NLE timeline doesn't really do anything as it doesn't recreate the data which has been removed in the first place.

You've missed my point. On a DV camera you don't have access to an uncompressed signal, so there's none to record. All you can do is make the most of it in post.

AFAIK, there is NO uncompressed camcorder available - none. It's just not needed. Uncompressed is great in post when you're moving between apps and need to avoid generation loss, but during shooting, it's a very expensive luxury.

Graeme

Ah indeed I did. My apologies!

I know that there is some (near finished?) work on a mod of the DVX100A that will allow it to export uncompressed video straight off of the CCD array and several HD cameras in dev.

<<<-- Originally posted by Aaron Shaw : Ah indeed I did. My apologies!

I know that there is some (near finished?) work on a mod of the DVX100A that will allow it to export uncompressed video straight off of the CCD array and several HD cameras in dev. -->>>

And that's just about the only way to do it in DV. But considering the cost of the camera involved and the problems inherint in messing with the sensor, it's probably more worthwhile just to buy your own sensor, like Obin is doing.

Bill, if your worried about degradation in post for filmout, you should read (and for all I know, you already have) the American Cinematographer article on 28 Days Later.

As I remember it, they took all of their 'printed' takes, upsampled them to 4:2:2 and put them onto D1, which is what they worked on in post and for color correction and their final online edit. The final negative, I believe, was scanned out from the D1 tape as well - after capture they never went back to DV. That can be expensive, of course, but if film out is your goal, it might be something to look at.

For going out to film, you should boost to 4:4:4 via a reconstruction, rather than a linear interpolation technique - as I have proved, it looks superior. And 28days was over sharpened, and poorly uprezzed. You'd get better results up-rezzing to an HD format using modern techniques rather than going to D1, which is a 4:2:2 SD format. 28days is no best practise in this regard....

Graeme

Interesting. I had no complaints with 28 DAYS LATER, considering it was shot on miniDV, but if there is a better way I don't have a problem with that! And I know Graeme is quite learned on this topic. Joshua, thanks, that is the way I understand this workflow too, but would like to add (you might find this obvious) that there's little point if any going back to DV after working in 4:2:2 or 4:4:4

Graeme, where can I/we learn more about the 4:4:4 reconstruction method that you mentioned?

http://www.nattress.com/filmEffectsGNicerTests.htm

Shows the results from my filter for FCP. I'm working on an improved version now....

Graeme

Any chance your plugin will be ported to Premiere Pro? I would love to get my hands on a copy...

I think the original poster may have been talking about a method whereby using Avid software, and possibly others, you can capture 10-bit SDI video, uncompressed, through the firewire port of a computer.

However, you still have to have multiple drives, probably, to support that level of data rate.

Also, it's irrelevent with any DV camera I know, none of which support an SDI out signal. However, it would be relevent with DigiBeta cameras or others.

Some DV decks support SDI output as well. They transcode the DV signal from the tape to an upconverted SDI signal. This process, as far as I know, would not improve the quality of the original signal in any way, however, neither would it degrade it.

Technically, no upconverted DV signal would be "improved" since the software can't add information that wasn't there in the first place. However, it can be altered so that it looks better for large format display, using software discussed above.

I, personally, think that video sources of all qualities, from DV through DigiBeta to HD formats can be improved though the use of clever algorithms - indeed, that's my current mission in video programming!! I wouldn't be talking about the subject unless I passionately believed it to be so!

I think those who've seen the current results of my research can attest to the veracity of my claims, but I myself think that I can still improve on what I'm doing now as I get more used to working in this problem space.

When a deck converts DV to SDI, it just linear interpolates the chroma to produce the 4:2:2 signal from the 4:1:1 signal. You're right to say that this doesn't improve the quality at all - just represents it in a nicer fashion. I, however, have developed a method of intelligent luma based analysis that takes all the known information into account - luma and chroma, to make a much more accurate guess as to what the original 4:4:4 chroma was before it was compressed. This is not magicing information out of nowhere, but making the most possible use of all the information we have, by using the full resolution of the luma as a guide to help reconstruct the chroma. As far as I know, nobody else is doing it this way.

Similarly, I'm working on uprezzing DV to HD, and again, I'm not using standard interpolation filters like bicubic, or specialist ones like sinc or Lanczos, which, after all, don't produce the best of results even in the best of circumstances. They're slow, cause ringing and are fairly dumb in that they don't understand the nature of the individual picture they're uprezzing. Some have attempted to improve matters using content adaptive scaling, which, again, AFAIK, manipulates the interpolation parameters on a pixel by pixel basis to improve the results - and they do, to an extent. Algolith make a great scaler using these techniques, for instance. My technique is completely different and doesn't involve interpolation filters at all, and is, to my eyes, producing much better results. I've also seen some comparisons between it, and results from S-Spline which is often mentioned on these forums. I can safely say, that so far it's looking to be a lot better and faster than S-Spline, but again, that's my eyes!

So what I'm really saying, is that you can make DV look better than it is, but just capturing it uncompressed via SDI alone is not going to do this. To say the native DV image is as good as it gets is true enough under normal circumstances, but doesn't take into account the exciting developments in image processing that I and others are working on.

Graeme

Hello,

I thought I would jump in on this thread...FireStore will only record an incoming DV25 signal from a FireWire 400 device such as a camcorder, switcher or deck. FireStore does not have a codec, nor does it have the ability to record and format other data streams other than DV25. The power of FireStore is that it is able to record a DV25 stream to a computer compatible FireWire drive in native NLE file formats. The processor and DTE Technology in FireStore makes it possible to write to disk in these native file formats. The end result is clips on disk that are identical to what your NLE would write during the capture process. This ensures that the clip is fully compatible with all the NLE's functions. Being on FireWire drives, it is possible to hot plug the HDD to Macs or PCs. All of this combines to eliminate the chore of ingesting your footage in the production process.

There are no plans currently for an uncompressed FireStore, mainly due to the fact that there is a small number of camcorders with uncompressed outputs and a lack of portable disk drives that are fast enough to record the stream. Remember, most FireStore products are portable devices designed to be used in the field.

But in the future, anything is possible. Focus has joined the HDV consortium and we hope to implement this format in FireStore products in the future. We are not yet sure in what form however. HDV is still in its early days.

On another note, the AJA Io box is the only device I am aware of that will transmit real-time SDI over FireWire 400. It of course, it requires a disk array to record the 270Mb/s stream and a Power Mac G4 or G5.

Hope that helps,
Matt McEwen