I'm doing a batch script to copy files from card to pc (joining spanned files) and convert to intermediate format (and/or create "down-sized" SD version as well)

how you guys edit the videos from ae50? native editing? convert to some intermediate format?

i use Canopus HQ for a long time for intermediate/export purposes, and a while back the codecs became free, so now is my solid choice.

also, if wanted, i pretend to release the script after testing and making a proper installer.

i started this because i like to came home from the event and go to sleep. so the script do "everything" unattended. (detect memory card > ask for output folder > copy and join files to disk > verify file count > eject memory device > convert to intermediate (HD) convert to intermediate (SD). its also "multi-threaded" (multiple encoding's at same time)

I edit native in premeire pro. I see absolutely no point in using an Intermediate codec. It is a unnecessary time wasting step for me..... obviously if you are a film maker you may need to for color correction purposes. But i find h264 stays together quite well with slight tweaks....

I edit native in premeire pro. I see absolutely no point in using an Intermediate codec.

I do work with Edius and edit natively which is a decission mainly because of space on the harddrive I"m trying to save (Canopus hq files can be 3 to 4 times the size) and that works out well, everything I do is realtime, including colorcorrecting BUT if space wasn't an issue I would convert every project to canopus HQ, the workflow is much faster in that codec, everything, including scrubbing the timeline, is butter smooth. You have to experience it to see the benefits.

Marlon: That is very cool, are you releasing the batchscript for free? Too bad it can only do one card unattended, would be great if you could attach several cardreaders at once and it would convert one by one.

i use vegas, editing AVCHD is good, not amazing, but good. the main problems is with long projects, start to become sluggish. with intermediate files everything is ultra-fast. i also use external encoder, so exporting is also very fast.

yes, i will release for free. just need more testing and testing. i think i can "implement" multiple card-readers also ;)

Marlon, would it be easy to modify the script to perform some more basic copy paste functions? I was thinking of attaching several cardreaders at once (as after a wedding there are always several cf cards to be copied) so that each cardreader gets a drive nr and then let the batch software run to copy all the data from the cards to 2 different drives of which one would be a backup?

How much larger are the canopus hq avi files compared to the native avchd files?

yes, i already did last night more than one card reader function. just need to implement in the whole thing.

backup function i was thinking also. maybe i put a function to ask if backup is wanted, so it copy the video only, merging the spanned videos to one place, and the original files from card + playlist + CPI files to another place (keeping original files as backup instead the processed ones, a.k.a. "full card backup")

canpus hq have normally 4~5x the filesize at 24mbps (about 50~60gb hour)

the convert functions will be optional also. i started this just to be a automated copy script, but the spanned files where a problem (files past 2gb). so i found a way to "analyze" the playlists in the AVCHD folder, detect which files are spanned, and join them directly on copy to disk (and after copy, rename to original recording date/hour for better organization and management)

I work with the good old Final Cut Pro 7.
I convert all the material to ProRes LT and work on it fluidly with my 6 1/21/2 years old MacBook Pro.

I consider changing to Premiere Pro or Avid MC as I don't like FinalCut Pro X.

With Premiere Pro CS6 I can just import the material without transcoding. But it's not really fluidly running if you start to add lots of footage or some FX. I know I'll need a new computer soon, but the camera put a hole in my pockets. No money left...

great stuff Marlon, will be very interested to see the end version.

thanks for the input guys. will release when ready ;)

Weird no-one else responded to this, seems there is not much interest in such a piece of software? I think it would be awesome to plug in all my cards in several readers, start up a batchscript and go to sleep to find all my data on my harddrive with a backup when I wake up.

maybe not everyone have a large amount of events/number of cameras to make a big deal out of this.

i also don't have that much need for automation, but the fact is, i automate, do shortcuts in everything i can to speed up workflow without compromise quality, that i can now see the result: "more free time" ;)

i do shortcuts for multiple vegas filters (plugin manager) i do a bunch of scripts to speed work, i did a automated encoder for dvd (convert exported file to video + audio in few clicks), scripts to scan video for black frames, batch to record dvd-r with single click, etc...

lot of time wasted to make them, but results are amazing ;)

With FinalCut Pro things are so easy - no need for further scripts.

I copy the card to my backup drive.

Then import into FinalCut, where all clips are transcoded to the codec I choose. With my old labtop, that needs some time, but I can go sleep...

Edius can do this too but that's not what Marlon script does, it appears to go futher then that, if I understand it right it should be possible to connect several cardreaders to your pc, put all your cards in it that you used at a wedding and start up the script that could take the data from each card, join spanned clips, and move all footage from all cards to a folder, or convert to a intermediate codec and then move the entire folder structure from each card to a sepparate folder/harddrive as back up, all while you are sleeping. :)

Well, that is what I would consider a major timesaver if Marlon could pull something like this off, I wouldn't even mind paying for such a script.

Sounds great - but only in the windows world which I left happily years ago...

"only in the windows world which I left happily years ago"
Watch out, this is how windows vs mac bashing usually starts, better to concentrate on the good stuff, no matter what OS you are using. :)

i think i will release simpler versions first to test it, then later implement encoding functions.

will be easy to debug if i start simple.

will start with multi-card/copy/merge/rename/backup functions, then later add the extras ;)

OBS: the playlist format (required for parsing) looks like the same for all NEX cameras, but i can only test with nex-f3 and nex-ae50 (when arrive, still waiting ;P ). probably will not work with "non-nex" files.

Marlon ~

Regardless of what anyeones says keep going. This sounds like a great time saver. Like someone else said there are people out there who need this sort of functionality and would be willing to pay for it.

Chris Young
CYV Productions
Sydney.

first functional version is ready. still didn't implemented backup and multicard-reader functions yet, but the conversion process is working fine.

for now, you can run it and select if want to copy files to hard drive only (merging spanned files and renaming to the video date), or copy and convert to Canopus HQ in HD or SD or both at the same time.

also added right-click option in folders so you can just convert already copied files in the hard drive to intermediate/sd

i got a few ideas, but need more info. how you guys do backups of the video? cause i use one external HD and internals ones. (copy all videos to external and internal to have it in 2 places).

who use external backup, do you have more than one or a single one? how is your folder structure? just a folder on the root of the drive?

im recording in Full-HD 60p, to deliver in DVD (30i) and Blu-ray (30i also). how you guys deliver the video? that is the final format/video specs?

I create a project directory and sub directories for each camera, use Sony software to transfer from cameras to appropriate directory then backup the whole to LTO3 data tape . When I finish my editing I delete from the hard drive. Do the same for the other edited files including DVD files and Bluray. Like Noa I use Edius for my multicam editing native AVCHD and Vegas and Sound Forge for audio.

Ron Evans

Hi Marlon!

Is it possible to copy files to hard drive only (merging spanned files and renaming to the video date) and right after that convert all to HQ HD and/or SD? (without the extra rightclick to select the option when it's on the drive.
I think for rough edits on multicams (I use up to 4 camera's now) it might be good to have it batch copy and after that convert to a intermediate of choice and let that run over night. Then edit in a hq sd file for the first rough cut and connect native files again once you start colorcorrecting. Editing 4 HQ SD streams should be space saving and edit like a knive through butter.

Currenty I back up the contents of the sd card with sony's software to a "import" drive and then back that up to a internal "edit" drive, I make another backup from that edit drive including all projectfolders to a external drive at the end of the day as well.

Ron Evans, LTO3 data tape works as a "hard drive-like" or its something that need proprietary software or something to transfer?

Is it possible to copy files to hard drive only (merging spanned files and renaming to the video date) and right after that convert all to HQ HD and/or SD? (without the extra rightclick to select the option when it's on the drive.
yes, you can copy only or copy and convert, no interation needed

I think for rough edits on multicams (I use up to 4 camera's now) it might be good to have it batch copy and after that convert to a intermediate of choice and let that run over night. Then edit in a hq sd file for the first rough cut and connect native files again once you start colorcorrecting. Editing 4 HQ SD streams should be space saving and edit like a knive through butter.
SD files are a bit smaller than original files (original 25-28mbps, converted about 22mbps) and yes, editing is very good ;P

i previously did a few proxy scripts for vegas, and they work perfectly. a few friends i have in the Brazilian forum edit h.264 videos from DSLR, and the video have a very high bitrate (about 44mps) making decoding even more slow than AVCHD at 25~28mbps.

the great thing is i did a script (for sony vegas) to change between files in the timeline: one click you can change the smaller ones for the original, one click again can change to proxy/sd/whatever wanted, in the middle of a project, without screwing up anything. maybe i can make it more friendly as a proxy creator also.

Currenty I back up the contents of the sd card with sony's software to a "import" drive and then back that up to a internal "edit" drive, I make another backup from that edit drive including all project folders to a external drive at the end of the day as well.
i was thinking about something like this: a marker file to make the program identify the backup drive and copy everything there for backup. it can be internal or external drive, so, even changing the drive letter/usb port/multiple external backup drives it can identify the connected one and backup there, so no need to specify it each time.

Backup tape drives need special software as they do not appear in Windows. I use Retrospect 8.1 as the backup software. Unlike Noa I am happy to edit native files so really have no need for an intermediate at all. I am interested in saving time transferring from 4 cameras I normally use for the events I shoot, even two at a time would be great. It really doesn't take me that long now though. I shoot theatre so projects are about 2 hours in length. It now takes about 12 mins a camera using the Sony software so less than an hour to transfer from the 4 cameras.

Ron Evans

nice, i will implement multi-card readers and backup option as soon as i can ;)

Ron, it's not that I don't like editing native files but if I"m adding 4 streams of native footage in a multicam sequence editing in a intermediate format makes it a much smoother experience, no matter how powerfull your system is. The biggest gain you will see is when fast scrolling your timeline. Once I have done a rough cut I delete the intermediate files and relink to the native ones for color correction and exporting. Also exporting hq files to a hq master is much faster then native files, on my older system I can export a one hour timeline under 10 minutes. Only if I would edit with hq sd files I would need to relink to the native files to be able to get HD files if needed or to go to blu-ray. I would only be interested in editing this hq sd because of the space you save and the visual quality is more then enough to do rough cutting and multicam editing is butter smooth in that way.

I don't scroll much as I effectively play the theatre show and switch cameras. Filters off while in multicam runs smooth enough on my i7 2600K system for 4 tracks of AVCHD and 1 track of EX3 all native. At most I run a slightly fast pace scrolling with my Shuttle Pro V2. I do all the colour balancing etc before I start and rarely need to do much afterwards to match some edits that do not quite work. I too export a HQ file to go to TMPGenc for DVD encoding. I tend to work in 5 min segments, switch out of multicam and watch what I have edited and fine tune the filter changes then if needed. Encode for Bluray with Quicksync straight from timeline. Do most all my authoring in DVD Architect 5.2

Ron Evans

I fast scroll all the time which saves me quite some time going quickly through the edit and choosing camera angles as I move on, once the rough edit is done I let it play from the start and finetune.

Looking forward to try the conversion tool Marlon. :)

I need to listen to the dialogue in a play or the beat for a dance show so really do not scroll fast unless it is a monologue or solo dance. I have settled on this short 5 mins or so segmented way of doing things that is effective for me. It is a serious retirement hobby for me so that does make a difference !!! I can finish a 2 hour show edit in a day. I may make a few changes after I watch the DVD so usually cycle the finished DVD twice before I am happy. Then just make the Bluray as the authoring is the same in DVDArchitect just substitute the Bluray file and change properties.

Ron

backup its now working.

just need to create a text file named "BACKUPHD.TXT" in the root of the drive, so the program will identify it as backup drive. if wanted, just put the folder name inside the text file (like "NEX BACKUP"), so the backup will be created in that drive, in the NEX BACKUP directory instead of the root folder (like if you use that drive for something more than video backup, so its more organized).

now the program ask for the filename of the destination directory in the selected folder. this remove the needed to manually create the folder on both drives before copying the files, making handy and simpler

if the program detect a drive with the BACKUPHD.TXT file, it present the backup options (yes/no) for those who don't do backups.

now for the multicard functions... (this will be "tricky" ;P )

Everything done. file copy function, encoding, backup, multi-card readers (up to 6)

just need to add option to install codec in the installer (if encoding function is wanted) and more testing.

Very nice Marlon! Can't wait to try. :)

well, its done.

be warned to test only in non-critical videos. im not responsible for any damage it may cause. use it "on your own"

ive made 2 separated installers, the codec and the program itself. if you do not want to do conversion (to intermediate, HD and SD) do not need to install the codec (Canopus HQ codec, x32 and x64)

first, if you want the option to backup all the card to a drive/external drive, create in the root of the backup drive a file named "BACKUPHD.txt" (if windows is set to hide extensions, will be BACKUPHD only). if you want the backup be made inside a directory in the backup drive, just type the folder name inside the text file (like "AE50 backup folder", with no quotes)

install the program

put the cards on the card readers (up to six) all together.

start the program on the start menu shortcut (AutoAVCHD v0.4)

it will show found drives, and it will ask for the output folder. this is your "work folder" or hard drive (it will create later a folder inside)

then choose if want only to copy files, with no conversion, or copying plus conversion to intermediate

if you created a backup file in the backup drive, it will ask if you want backup

then, type the folder name for the event (example: 2013-01-01 new year video), press enter

now sit and relax. i mean, see if it is working properly ;P

if file conversion was choose, it will encode after everything was copied.

verify output folders to see if everything is OK.

OBS:
to choose the options on the program just type the number of it and press enter.

do not use uncommon characters, preferably alphabet + dot, dash, space, underline.

tested only in Windows 7 x64, but should work on Windows 7 x32 too.

it only work with SD cards, not memory stick cards. the folder structure is different, later it will work on both cards type.

if you selected only to copy the files, but later you want to convert to intermediate, just right click on the desired folder, and go to "AutoAVCHD". it will give options to convert already copied files on the HD. (also is recursive, so it can convert all cards folders inside the main folder)

if any problem occur, please be descriptive about it. "didn't work" does not help.

AutoAVCHD:
AutoAVCHD_v4b2.rar (http://www.mediafire.com/?00ng27q0pag947a)

Codec:
AutoAVCHD_Codec_v0.1.rar (http://www.mediafire.com/?307e6io5fc0r9r1)

Great! :) could it be that cards from a canon 550d won't work? It didn't recognize the card, it did work with a card from my ea50 but I got a pop up error message first which said it couldn't find the drive and it mentioned "device\harddisc\dr6", I had to press close several times to get rid of this notice, after that I saw another box appear asking for the location of the files and from there on it worked.

What I wanted to ask, after the file transfer, can it do auto back up to another drive?

Thx for all the hard work :)

it only work with sony nex cameras. DSLR i believe you don't need something special, cause the file splitting is independent, so just copying is enough.

about the popup notice, i believe is due to empty card readers. i will try to find another method to detect drives that don't cause that ;)

if you created the BACKUPHD.TXT file i mentioned in the backup drive, it will do backup before file copy to the "main" drive. did it say it found the backup drive before asking the output name?

new version!

AutoAVCHD: (b3)
AutoAVCHD_v4b3.exe (http://www.mediafire.com/?66h8rovburfdieo)

Codec (v2)
AutoAVCHD_Codec_v0.2.exe (http://www.mediafire.com/?6fuxswk28o96yyj)

uninstall the codec and the application if you had installed previous versions before installing newer ones. (uninstall via "programs and features" applet in control panel, the start menu shortcut had a bug)

Changelog (v2 and b3):
# Fixed file encode mode (path problem)
# Fixed Uninstall shortcut (both programs had the same one, now separated)
# Fixed errors when scanning drives (new method used, no more errors on empty card readers)
# Fixed codec in Windows 7 x32
# Now tested in Win7 x32 and Win7 x64

I work with the good old Final Cut Pro 7.
I convert all the material to ProRes LT and work on it fluidly with my 6 1/21/2 years old MacBook Pro.

I consider changing to Premiere Pro or Avid MC as I don't like FinalCut Pro X.

With Premiere Pro CS6 I can just import the material without transcoding. But it's not really fluidly running if you start to add lots of footage or some FX. I know I'll need a new computer soon, but the camera put a hole in my pockets. No money left...

Hi Erik, I too am using FCP7. Although I have recently been using clipwrap to transcode to Pros Res LT due to FCP7's inability to transcode 1080 50p. What do you archive? Do you keep the pro res quicktime files and the AVCHD PRIVATE folder?

new version!

I still have to test it but will let you know my findings as soon as I can.

Hi Erik, I too am using FCP7. Although I have recently been using clipwrap to transcode to Pros Res LT due to FCP7's inability to transcode 1080 50p. What do you archive? Do you keep the pro res quicktime files and the AVCHD PRIVATE folder?

I am in a kind of testing phase with my EA-50 - as I always was with DSLR videography.

I am definitely not that happy with the codec on the EA50. Thats why I convert to ProRes. I like to have a good codec for editing, but converting means a lot of time for me. I archive the ProRes files but hold the AVCHD folder structure too.

At this time I think I'll switch to another workflow this year. I have buy a replacement for my old MacBook Pro but am still not sure about what to come.

Sorry, but I am still not to confident about my workflow to recommend for anybody else. And it seems I am the only one not filming weddings at all... ;)

Erik, I suggest you look at Lightworks v11.1 as it is the fastest yet editing native AVCHD. However, like you I prefer to transcode AVCHD to ProRes or better still, Avid's DNxHD 185 10 bit as it works faster on the PC.

Lightworks is in 'beta' on Unix and on Mac very soon too, then all three will update together.

I suggest you look at Lightworks v11.1 as it is the fastest yet editing native AVCHD

Is there a benchmark out there with all existing NLE"s to back up that claim? :)

Well Noa, Lightworks is one of the few NLEs apart from Avid used by professional editors to edit Hollywood movies, not weddings. But if you read what I actually wrote, you will see that I referred to the new version v11.1 being the 'fastest yet to edit Native AVCHD' That is: the fastest version of Lightworks to date.

Ah ok, that clears things up, allthough I don't understand the reference with weddings and hollywood movies if it regards to avchd editing, surely hollywood productions aren't filmed in a avchd codec :) If lightworks allows avchd editing I take it it targets the consumer market as well.

Well Noa, Lightworks is one of the few NLEs apart from Avid used by professional editors to edit Hollywood movies, not weddings. But if you read what I actually wrote, you will see that I referred to the new version v11.1 being the 'fastest yet to edit Native AVCHD' That is: the fastest version of Lightworks to date.

Do you know how it compares running 5 tracks of AVCHD with native files like I do with Edius realtime.

Ron Evans

It doesn't matter really which one is faster, I just misunderstood him when he said lightworks was the fastest for avchd editing, that's all. Must say that Craigs comment about lightworks being for pro editors dealing with hollywood productions and not weddings sounded quite condescending, maybe I shouldn't have bothered to react so we could have go back to the topic of this thread; dealing with ea50 files natively or as a intermediate file..

Please don't misunderstand me. I do not want to sound rude or condescending but I think some accurate information is required here. The more information we have, the better we can make informed choices.

Lightworks is not some new kid on the block. It has been a popular albeit very expensive high end NLE for over 20 years but it was designed around the standard broadcast codecs of the time. You've all heard of DaVinci Resolve? Similar to Lightworks, it too used to be an $80K 'turnkey' broadcast system until Blackmagic Design bought it and decided to update it then sell the software only for $995 (plus a free 'lite' version)

Similar idea. Editshare in the UK acquired Lightworks through a takeover about two years ago then set about updating it's codec platform before releasing a Free version: which requires transcoding of consumer formats and a Pro version for an annual subscription of only $60. This small yearly fee includes all future updates, professional codec license fees eg: Apple's ProRes, a simple but effective title generator and a greater choice of Export codecs. I should point out here that Lightworks does not set out to be an 'all in one box' post production tool. It is first and foremost, a fast and fluent editor with a very refined code. This is one reason many editors migrating from other NLEs appreciate the development which has taken place over the years.

With the recent release of v11.1, Lightworks Pro now includes native H.264/AVCHD editing without transcoding as this feature has been a popular request on the Lightworks Forum. Naturally enough, when you offer professional software to a wider audience at an affordable price, those operating with DSLRs and prosumer cameras will be a large and vocal group.

Craig, the list of known issues and problems in the read me file make it almost unusable in lots of cases as far as I can tell. It will not record HDV over firewire, pretty basic, AC3 audio will not import on mov files, no audio linking for AVCHD with AC3 audio, XDCAm ( mov ) Quicktime files will not play correctly etc a long list that for people using AVCHD with AC3 or some XDCAm users makes it an unusable piece of software even if it is free. I have downloaded the free version and will try to see how it works.

Ron Evans

Ron, you would do better to download the Pro 64bit version whilst the 30 day free trial is in place as the Free version has only very limited codec support. That is, with only the 'Free' version, you will need to transcode first as license fees for most popular codecs need to be paid - eg: to Dolby Labs for AC3 as they 'own' it.

Note: If you need AC3 audio support in Lightworks, just install the 'AC3Filter' and configure it for stereo or 5.1 surround.

PS: Do watch the video tutorials first as it will make your evaluation more informed and may I suggest you direct any Lightworks specific questions to the Lightworks Forum, which I'm sure you will find extremely helpful.

Yes I have downloaded the Pro64bit version and will look at the tutorials.

Ron Evans

Craig, I have had a week or so now to look at Lightworks and see the background in film editing clearly. The asset management is clearly a strong point as would be the case managing film stock etc. For someone used to the modern approach of dragging assets from explorer to the timeline and exporting it seems a little dated. My projects are all multicam and I found the multicam convoluted compared to either Edius or Vegas. The marketing also seems to be ignorant of competitors capabilities too. I really didn't see anything unique that wasn't in the editors I have, CS6, Vegas and Edius The user interface is a lot cleaner in these editors too as the Lightworks interface looks like it was designed for much lower resolution monitors , which might have been needed several years ago. I think that LIghtworks is playing catchup to the modern NLE's it is clearly a solid performer but lacks some of the modern looks. I will stay with Edius and Vegas I think.

Ron Evans

Ron, each to his own I guess but I would disagree on a couple of points. Even though Lightworks offers basic colour correction, dissolves, wipes and DVE moves, it is principally a professional 'editor' where most 'effects' are still commissioned out to third party specialists. Current Lightworks users seem to fall into two distinct groups:

1) Professional programme and feature Editors working in a 10 bit domain who generally Deliver an EDL or a DPX sequence on to other specialists.

2) hobbyists and semi professional users who Shoot, Produce and Deliver in the 8 bit domain.

There are probably a lot of features and shortcuts you missed in only a short evaluation and as I wrote before, Lightworks software was designed around a dedicated hardware control surface (the Console) and in my opinion, any evaluation performed without at least using a Contour Shuttle Pro 11 or similar would be a pretty futile exercise. Lightworks will 'work' with a mouse but it immediately becomes 'alive' with a custom programmed Jog/Shuttle device.

I note on the Lightworks Forum many editors migrating from Avid Media Composer over to Lightworks and their reaction has been the same: once they become accustomed to Lightwork's different methods, the big plus is speed of editing at a very low subscription price point. I have posted your comments on the Lightworks Forum (anonymously, under my avatar) so that others may offer their opinions and reactions to your evaluation as many of the regular Forum contributors also work with CS6, FCP and Edius.

PS: To preserve all possible 8 bit picture quality, I'm currently performing all my post production in the 10 bit domain. My current 10 bit AVCHD Lightworks workflow is:

1) Transcode 8 bit AVCHD to 10bit DNxHD 185 with timecode using AWPro
2) Set up a 10 bit Lightworks Project and edit.
3) Export a 10 bit DPX sequence Master video file
4) Export a 16 track 24 bit WAV file Master audio file

5) For BluRay 5.1 Delivery, I Export an uncompressed 8 bit YUYV sub master AVI file with twelve to sixteen separate audio tracks, bypassing Lightworks internal audio mixer. This is then imported into Vegas Studio for mixing to 5.1 Surround and burn to BluRay disk.

Craig
I need more than a simple editor for my work and I do indeed use a Shuttle Pro V2 set up so that it works the same way in all the programs I use since they all allow custom setup of these control features. My main programs are Edius, Vegas, DVD Architect, AE, Photoshop, Sound Forge, TMPGenc etc so my work flow is easy. I like working with a combination of the Shuttle and the mouse and rarely use the keyboard. The custom keyboard was something I used way back with the Fast Video Machine and I have moved on from there. Shuttle in left hand, mouse in the right hand is very fast. I agree to each is own but I see nothing special in Lightworks that would make me want to look any closer.

Ron Evans