Partially Filled CF Cards

[Ronald Jackson]

A couple or three basic questions from this newbie.

Can I remove a partially filled CF Card from my nano, import into FCP (via card reader) and simply re-insert card into nano and carry on using "on top" of footage previously recorded?

Can I also export an edited sequence from FCP to a formatted CF card (via card reader) and carry on using the card "on top" of the imported footage?

Can I also export several separate edited sequences to a CF card?

Just starting out with nano, FCP (and a Nexto device) so am a bit like a cat on hot bricks until and if familiarity kicks in.

Ron

[Dan Keaton]

Dear Ronald,

Can I remove a partially filled CF Card from my nano, import into FCP (via card reader) and simply re-insert card into nano and carry on using "on top" of footage previously recorded?]

Using normal techniques, the answer is no.

The Mac, and Windows 7, and most likely Windows Vista, write extra files to the CompactFlash card when the CompactFlash card is accessed or used in one of these systems, causing problems when the card is re-inserted into the nanoFlash.

Note: The Nexto DI devices do not write these extra files to the cards and thus your procedure works great with the Nexto DI 2700 and 2500.

Background: The nanoFlash takes great care to not create fragmented files on the cards, and uses other techniques to achieve very high write performance. When other systems write to the cards, then write performance suffers when the card is re-inserted into the nanoFlash.

For this reason, we require that the cards be formatted in the nanoFlash before the first use, and after the cards are re-inserted into the nanoFlash after being in a computer, but not a Nexto.

Can I also export an edited sequence from FCP to a formatted CF card (via card reader) and carry on using the card "on top" of the imported footage?

No, for the same reason.

Can I also export several separate edited sequences to a CF card?

Yes.

Just starting out with nano, FCP (and a Nexto device) so am a bit like a cat on hot bricks until and if familiarity kicks in.

Ron

Great, we are here to help you get the most out of your nanoFlash.

For best results, just record the files on the nanoFlash, copy the files to a computer, then reformat the cards in the nanoFlash before recording again.

Deleting files from the cards, then reusing the cards without reformatting them again is not supported.

We could support this technique, but then the nanoFlash would be limited to much lower bit rates.

[Piotr Wozniacki]

Originally Posted by Dan Keaton

Deleting files from the cards, then reusing the cards without reformatting them again is not supported.

We could support this technique, but then the nanoFlash would be limited to much lower bit rates.

I'd add to what Dan has outlined above something from my my practical experience:

- I can read / copy from a partially filled CF card in my Vista PC, then reinsert it to the nanoFlash and continue recording (though I realize the performance of the card might be affected - at the moderate bit rates I'm using, I never notice this though)

- I can not delete files (write operation) on my PC, and continue to use it on the nano without reformatting - though generally it seems to work, it cannot be trusted as for instance, the bar showing card usage (at the top of nanoFlash display) is not reset.

I guess this might be one of the reasons of card hot-swapping still not being possible, am I right, Dan?

[Dan Keaton]

Dear Piotr,

The issue is just file system fragmentation.

The whole design of the nanoFlash, in order to achieve very high bit-rate performance, is to avoid file fragmentation.

Hot Swapping is something entirely different.

When one inserts or removes a CompactFlash card in the nanoFlash, we have quite a lot of work to do, to prepare for reading from or writing to the card and testing the card to ensure that it is one that will record properly.

With Hot Swapping, we have to do (about) the same things, we just have to do it while the audio and video is coming at us at 1.485 Gigabits per second, while we are processing and recording the data, and opening and closing files on the fly.

We had done a lot of work on Hot Swapping.

I will just say, for now, "Stay Tuned".

[Piotr Wozniacki]

Thanks Dan - your "Stay Tuned" statement is rising confidence again :)

I notice the housekeeping tasks nanoFlash is performing at each card insertion, so I realize doing it while handling the incoming data is not a trivial task. But obviously, nanoFlash could do most of it just once with every individual card, and mark it as "ready" until the next formatting... This way, upon insertion, recording could start right away - just like it does now when automatically rolling over to the other slot at filling up a card (something the AJA cannot do).

Am I thinking in the right direction?

Edit: I should have written "mark it as "ready" until it's been recorded to again" and not "mark it as "ready" until the next formatting"...

[Dan Keaton]

Dear Priotr,

We discussed Hot Swapping yesterday with our team.

They feel that they are very close.

But, I do not have the details, at this time, as to how they are implementing it, sorry.

[Barry J. Weckesser]

Originally Posted by Dan Keaton

Dear Ronald,



Using normal techniques, the answer is no.

The Mac, and Windows 7, and most likely Windows Vista, write extra files to the CompactFlash card when the CompactFlash card is accessed or used in one of these systems, causing problems when the card is re-inserted into the nanoFlash.

Note: The Nexto DI devices do not write these extra files to the cards and thus your procedure works great with the Nexto DI 2700 and 2500.

Background: The nanoFlash takes great care to not create fragmented files on the cards, and uses other techniques to achieve very high write performance. When other systems write to the cards, then write performance suffers when the card is re-inserted into the nanoFlash.

For this reason, we require that the cards be formatted in the nanoFlash before the first use, and after the cards are re-inserted into the nanoFlash after being in a computer, but not a Nexto.




No, for the same reason.



Yes.



Great, we are here to help you get the most out of your nanoFlash.

For best results, just record the files on the nanoFlash, copy the files to a computer, then reformat the cards in the nanoFlash before recording again.

Deleting files from the cards, then reusing the cards without reformatting them again is not supported.

We could support this technique, but then the nanoFlash would be limited to much lower bit rates.

I am glad I came across this thread - ever since I got my Nanoflash (Sept. 09) - I have been doing what you describe above (partial download and then reinsert the card without formating). I have never had a problem. I am using now mainly Transcend 400 64 GB and Photofast 64 GB cards and record MXF 30p at 100 mbps. I have done this even with the cards filled to capacity and noted no problems.

What risks am I taking - loss of data? cards running slowly.? I have never reformatted a CF card until it is absolutely full - I just like to see what my shots look like at the end of the day and then continue on the next day using the same card.

[Dan Keaton]

Dear Barry,

Is your computer a PC running Windows XP.

If so, then, as far as I know, the computer is probably not writing any extra files to the cards.

Both of the card types you mention support up to 220 Mbps in the nanoFlash.

At 100 Mbps, there is some extra bandwidth available.

However, we still do not recommmend this practice.

The possible ramifications would be a "Card too Slow" error message, the stopping of recording, while we downshift to a lower bit-rate, then the resumption of recording.

[Barry J. Weckesser]

I am using a Vaio Laptop running Vista Ultimate - I make one large source file (addding to it day by day) until the end of the trip and backup on a G-tech Raid 0 drive as well as an accessory internal drive in the laptop each day, add the video files to my NLE Timeline (Edius 5.5) as I go along and then dump the the large source file (as well as the Edius project file) into my desktop video editing computer (at the end of the trip) using the G-tech with an eSata connection (had over 250 GB file from the recent Maui trip) - that way, in just a few minutes I can be up and running on the main editing computer with files already on the timeline and ready to be trimmed, filtered etc. I have been using this technique for the last year and haven't had an error (yet). I haven't had any "stopping of recording" messages or non-recording (that I am aware of). I guess the main reason for not reformatting is that despite 2 hard drive backups on the road and being able to scrub through the clips to look for errors with the NLE I still like the peace of mind of having the "original" files on a CF card. I know, I am probably being too compulsive with this.

[Piotr Wozniacki]

Barry - as I said above, I also do it quite often, but never with mission-critical stuff!

If Dan says this is not supported, and the CF card should be formatted in the nanoFlash each time before recording - he must have reasons for this (like CD's internal lab measurements of peak datarates, etc., etc.).

[Dan Keaton]

Dear Barry and Piotr,

I am just being conservative.

If your computer does not write to the card, then everything is ok.

With my personal Sony Vaio Windows XP computer, I never experienced this.

I skipped Vista.

I find that Window 7 writes to the card. Mac's write to the card.

If someone with mission critical files does this,
on a system that does write to the card,
using a card that just has enough speed to handle the bit-rate choosen,
or they have choose a smaller "File Size" instead of 100%,
then we have not done our job warning the customer, as it might fail.

Please note that all of this depends on the length of the recording.

We have a huge First In, First Out buffer built into the nanoFlash.

For short recordings, we can overcome a card that is not up to performace spec's since it is fragmented.

Also, CompactFlash cards can have times, while recording, where the performance is not up to spec, since it is internally "wear leveling" the card. Our large internal buffer compensates for this.

So, for mission critical files, we highly recommend formatting the card, in the nanoFlash prior to recording.

If using a lower bit-rate such as 100 Mbps or 50 Mbps, using a high performance card, one that we rate for 220 Mbps or higher, then it may just work fine.

We have built into the nanoFlash, starting with firmware 1.6.29, a special keystroke sequence that you can invoke that will show you how much of our internal "FIFO" is currently used.

(I will share the keystroke sequence with you as soon as possible.)

[Barry J. Weckesser]

Thanks for the input Dan. Do you happen to know if, in Windows 7, there are files written back to the CF card when you open up {CLP} and then highlight just the clips you want to copy and do a copy and paste onto the computer hard drive as opposed to downloading the entire card?

Perhaps your internal buffer has been what has kept any problems from happening with my shots since over 90 % are from a few seconds to maybe 5 minutes at the most (most under 1 minute).

[Dan Keaton]

Dear Barry,

I do not know when the files are written to the cards.

I will need to run some tests, but I am suspicous that they may occur when the card is ejected, using the normal safe "Eject Card" process.


I promised to provide the Keystrokes to enable the on-screen display of our FIFO buffer levels.

You must use the record button as the trigger (System|Trigger|Record Button), then simultaneously hit the record and left arrow to display the FIFO meter.

You will notice that the FIFO (First In, First Out) Buffer level will jump at times.

This is when we are simultaneously recording the video, while going back to close out the previous clip.
Or it could jump when a card is doing "Wear Leveling".

After this "Double Duty" is finished, the FIFO level will tend to go back down if the card is fast.

But, it may just jump to a new level and stay there until we have to close a new file.

But, if it jumps up to a new level and stays there, it is still ok, as long as it does not reach 100% before the end of the recording or both cards are filled.

Watching the FIFO levels gives one insight into what is happening internally, and allows one to evaluate their CompactFlash cards' performance levels.

[Billy Steinberg]

Originally Posted by Dan Keaton

We had done a lot of work on Hot Swapping.

I will just say, for now, "Stay Tuned".

Hi Dan,

This is encouraging news. As you may remember, I was posting a message every month, when a new nano firmware was released, asking you if CD was ever going to actually release hot swapping, even though it was "promised" for "next month" every time. After seven months or so, I figured I was just annoying you, and so I stopped asking. Now that more than a year has passed, should I start asking every month again? :)

(I won't, but I will mention again that hot swapping would be a VERY useful function for many of us, me included).

I've been testing the AJA implementation of the ProRes codec all this week. I'm shooting ballet for PBS using top of the line Sony Broadcast cameras [1500R's], at -3db gain. While the true comparison will probably wait until I get back home, since I don't have the nano with me, I think the 100Mb Long GOP 8-bit CD/Sony codec does as good a job for acquiring video as anything AJA has to offer. I still edit and grade in 10-bit, but acquiring in 8-bit with a nano still produces superb pictures that I'll stack against anything, including uncompressed.

Billy

Oh, and ps, would you PLEASE PLEASE PLEASE put back the two second delay before stopping a recording when hitting the stop button. It's way to easy to have a bad accident.

[Dan Keaton]

Dear Billy,

I tried to send you an email so we could discuss your request.

Could you please send me an email. I will be happy to call you.