An automated Vegas 24p workflow.. at DVinfo.net

Steve Szudzik · May 16th, 2007, 07:32 PM

We've been discussing in another thread here about automating the 24p workflow. Bill Fransworth and Mike Dulay came up with a a workflow using DGIndex and VirtualDubMod and some scripting to put it together, though there were still a few manual steps.

The thread and what needs to be installed is here: http://www.dvinfo.net/conf/showthread.php?t=92893

I put together a Vegas script DLL that further automates the process so you really don't have to do a whole lot of anything to remove the pulldown. The script will ask for a working directory (should be a clean directory that you don't mind the script to delete or copy things to) and the paths to the DGIndex and VirtualDubMod applications as well as the AVI Synth template. You can specify the script to parse the required audio delay as well, which it will gather from the AVS file generated by DGIndex. Another option is it can create a new 24p vegas project and put the output from VirtualDub into the timeline (so you can re-render to your choice of codec etc..).

Next is the compression used by Virtual Dub. I put a couple templates up there for Cineform, Lagrinth, Uncompressed AVI etc. Just because they are there doesn't mean you have them installed though, so make sure to pick one you like. If there is another one you'd prefer, you can choose the "custom" option and create the internals for Virtual Dub job script yourself. You just need to leave out the script calls for VirtualDub .Open, .subset, SaveAVI and Close. My script will add those automatically with the correct frame settings and file names.

The final option (second tab of the dialog) is the list of media clips found on the selected video track. Check the checkbox next to those that you want to run the script against. I strongly recommend just one track and one video clip for now though.. It shouldn't screw anything up otherwise, but hey, this is kind of a beta here..

So here is the script (my first one and a bit of overkill personally):
http://www.szudzik.net/vegas/scripts/24pWorkflow.html

Click save and place it in your vegas scripting folder.

I'm still going to work on this some more. Right now, it will process the whole file represented by each event on the timeline. It ignores clips, markers and all of that. I just grab the VideoEvent and get the underlying media file path and will operate against the entire file. As a result, it's best to just test this with a single track and a single video on the track.

The script dialog also allows you to load and save "template" scripts that you can use to update the script dialog for how it operates. This doesn't apply to the list of clips, but only to the general settings tab.

The "Clean" button on the dialog will DELETE ALL files in the working directory specified on the dialog. Make sure that's what you really want. I'm not responsible if you delete anything accidentally! Really, I'm not!

The Close button works as a "cancel" or "done", click it when you want the dialog to go away.

The "Run" button will start the workflow process. Depending on what options you choose and how big your file is, it might take a few seconds or significantly longer. Each of the resulting AVI's generated by VirtualDub will be placed in the working directory with the same name as the original video file, but with _24p.avi appended to the end.

I think that pretty much covers it for anyone who wants to give this a try. Like I said, try 1 clip first and see if it does what you want. I'm going to get a page crafted with the above info and some screen shots to serve as my "Help" file, hopefully done in a little while with that.

I'm going to comment the source code and will later make that available as well for anyone who wants to make changes or improve the workflow.

Again, this is "BETA" type software here, so it's a use at your own risk type of thing. I'll fix up any issues you guy's find and listen to suggestions for it as well though.

--Steve

Steve Szudzik · May 16th, 2007, 08:56 PM

Ok, I found one small bug (new I rushed it a touch) and have posted a quick updated DLL on the page. The bug caused the process to fail the first time through because it didn't detect the AVS generated by DGIndex. The second time through worked properly. It now works the first time as well as subsequent times.

--Steve

Mike Dulay · May 17th, 2007, 08:39 PM

Steve, you may want to update your instructions on how to invoke the script dll file. I was looking around Program Files when I realized you could store it anywhere. Ran into a roadblock with the Cineform HDLow default compression in the script.

How I ran:
1) Downloaded Vegas trial from sony
2) Installed and registered trial version
3) Dragged and dropped an m2t file unto the Vegas timeline (setup for HD size with 23.796fps)
4) Stored your szudzik.HV20Pulldown.dll unto c:\clip (arbitrary choice -- probably should be C:\Program Files\Sony\Shared Plug-Ins)
5) Invoked Tools -> Scripting -> Run Script... -> specify location of the dll
6) Specified c:\temp as my work directory
7) Specified C:\clip\24p\dgindex\DGIndex.exe as my dgindex path
8) Specified C:\clip\24p\VirtualDubMod_1_5_10_2_All_inclusive\VirtualDubMod.exe as my virtualdub path
9) Specified C:\clip\farnsworth.avs (this Farnsworth's original template with -222ms ... from here ... http://www.hv20.com/showpost.php?p=1421&postcount=10)
9) Selected Calculate delay audio
10) Left the default option CineformLowHD
11) Run

DGIndex runs okay, generates all the files in c:\temp, syl file is generated and virtualdubmod 1.5 launches. Now I get the immediate message "License Required: This application is not licensed for CineForm Hd encoding. Please visit www.CineForm.com to purchase your upgrade." I click okay and virtualdubmod continues to run. The generated avi file in c:\temp displays the cineform logo (too bad, the file looked good and was only 1x larger vs 33x). When this is dragged on to the vegas timeline you can see that the logo is hardcoded into the avi. I think you should rather default to Uncompressed (but recommend in the dialog LagarithLossless or HuffYUV codec).

I repeated 10-11 but with Lagarith this time.

12) I add back the 24p AVI file to the timeline ... rendered to HDV intermediate. I can probably use this as my archival format and edit format (but not as a playback) ... for final render I would use a common codec like AVC/H264 or WMV.

One thing I can comment, it is so clean from a use perspective! I think you'll help sell more copies of Vegas. 8-)

Steve Szudzik · May 17th, 2007, 09:16 PM

Thanks for the feedback Mike. Yeah, I just threw those instructions together quickly. I'll get them updated soon hopefully. I today also got the dialog from VDub about the cineform codec, guess my trial has ended ; ). It still won't uninstall but interestingly, I don't have the logo either. Maybe a bug with vista and how they uninstalled.

For everyone else. I have also made this "script" into a stand alone executable that you can use to automate and batch the workflow. You can find it at the below location:

http://www.szudzik.net/vegas/scripts/HV20Pulldown.html

It's C#, so requires the .Net Framework 2.0, just like vegas but his doesn't have any Vegas dependancies. The UI is nearly identical to the vegas script, so this page should look familiar.

The difference here is that one the 2nd tab, you can select all the media files, from wherever you want and add them for processing. This version also does not require the copying of the file to the "working directory" so it's much faster.

Let me know what y'all think..

Steve

Ajit Bikram · May 18th, 2007, 09:54 AM

Downloaded your exe file. Will report how it works once I get some 24p footage this weekend! ;-)
Thanks for all your effort and time!

Steve Szudzik · May 18th, 2007, 10:06 AM

No problem, hopefully it will work well for you. I might make a few more updates to the standalone app this weekend. I might "hide" the DGIndex window. It's such a short process, so you don't gain much by watching it work. I wouldn't mind hiding the VDub window as well, but since I can't get any progress info back from it, I'm reluctant to do that. At least with that window up, you can see what's going on.

I think I prefer the standalone app right now myself. The Vegas one can be good, but I'd have to put in a lot more work. It would be nice if it honored the splits and markers etc.. Someday maybe ; )

--Steve

Mike Dulay · May 19th, 2007, 02:04 PM

Steve, your standalone exe is working great. I've got a harddrive full of m2ts that I'm converting right now. Wish my new laptop was faster but hey, I'll take what I can get. I'll have to make a new blog referencing how to put your exe together with all the other freeware software. Can't have you doing all the work. 8-) Thanks for doing this!

Steve Szudzik · May 19th, 2007, 02:36 PM

Thanks Mike, glad that it's working well for you.

I just updated my documentation for the application at: http://www.szudzik.net/vegas/scripts/HV20Pulldown.html

Put in the links to the required apps as well as a better workflow on how to get started. I also made some small changes to the app last night. Got rid of the "clean, run, load, save" etc.. buttons and put them in a file menu. Looks a bit cleaner. Also doc'ed most of the code and should be posting that up shortly.

Funny thing is there's more code for working with the UI than there is just to run the workflow! Could easily be fully scripted in a batch file as well, except then it's tougher to select the video files, unless you just batch process an entire directory.

--Steve

Ajit Bikram · May 19th, 2007, 06:53 PM

I downloaded all the scripts and the self-executable file. Also had to install xvid. The program works and gives an output. But the output definitely doesnot look to be progressive!! (While playing I can see the combing/ghosting).
To test I did the following to compare....
1) Converted the m2t file to avi.
2) Opened the avi in VirtualDub.
3) Used the telecide filter
4) Played the output-- this definitely does not have those artifacts and is progressive. Looked at it frame by frame. But it does have one duplicate frame (every 5th) as it should.

So what I might be doing wrong???

Mike Dulay · May 19th, 2007, 07:35 PM

Ajit, are you manually doing the process rather than letting Steve's exe create the avi for you? The exe should have done the TFM and TDecimate for you already. #3 on your list should not be necessary as there is no longer any pulldown for you to process with telecide.

Ajit Bikram · May 19th, 2007, 07:41 PM

Hi Mike,
I tried doing it using Steve's exe. But that did not work out! That's what was my point saying that the output is not progressive!
However when I used the filters manually in virtualDub it gave me progressive.
can you think of any possible reason?

Mike Dulay · May 19th, 2007, 07:44 PM

Ajit, Steve's exe isn't standalone. You probably did not install the TIVTC plugin into avisynth. Look for post #26 on this thread ...

http://www.dvinfo.net/conf/showthrea...t=91500&page=2

Ajit Bikram · May 19th, 2007, 08:01 PM

Quote:

Originally Posted by Mike Dulay

Ajit, Steve's exe isn't standalone. You probably did not install the TIVTC plugin into avisynth. Look for post #26 on this thread ...

http://www.dvinfo.net/conf/showthrea...t=91500&page=2

I did understand that Mike. I was following the whole development from Farnsworth first post at the other forum.

Anyway I might have missed installing something. I am going to check the whole thing again. I do remember that I downloaded the filter but I have to check if I installed it properly.

I'll get back soon.
Thanks..

Steve Szudzik · May 19th, 2007, 08:26 PM

If the specific VirtualDub script that you selected in my app (I call them "compressor") doesn't quite do what your looking for, you can use VDub to get the settings how you'd like. Do that and the click "Save AVI" but instead of saving, tell it to create a job. You can then look at the jobs in VDub and save those to a file. Open the job file and you can then bring that in as a "custom" script on my UI. Just leave out the Open, Close, SaveAVI and .subset portions of the script.

It could also be something in the .AVS template file as well. I know that Farnsworth & Mike both have a few different settings between them.

Anything you can achieve by just using DGInex & VDub can be done in my app. I'll I'm doing is automating the files through those two applications.

--Steve

Ajit Bikram · May 19th, 2007, 09:32 PM

Can you tell me when I install the TIVTC plug-in, I just have to copy the TIVTC.dll file into the avisynth plug-in folder. Am I correct?

May 16th, 2007, 07:32 PM	#1
Steve Szudzik Regular Crew Join Date: Mar 2007 Location: Sammamish, WA Posts: 103	An automated Vegas 24p workflow.. We've been discussing in another thread here about automating the 24p workflow. Bill Fransworth and Mike Dulay came up with a a workflow using DGIndex and VirtualDubMod and some scripting to put it together, though there were still a few manual steps. The thread and what needs to be installed is here: http://www.dvinfo.net/conf/showthread.php?t=92893 I put together a Vegas script DLL that further automates the process so you really don't have to do a whole lot of anything to remove the pulldown. The script will ask for a working directory (should be a clean directory that you don't mind the script to delete or copy things to) and the paths to the DGIndex and VirtualDubMod applications as well as the AVI Synth template. You can specify the script to parse the required audio delay as well, which it will gather from the AVS file generated by DGIndex. Another option is it can create a new 24p vegas project and put the output from VirtualDub into the timeline (so you can re-render to your choice of codec etc..). Next is the compression used by Virtual Dub. I put a couple templates up there for Cineform, Lagrinth, Uncompressed AVI etc. Just because they are there doesn't mean you have them installed though, so make sure to pick one you like. If there is another one you'd prefer, you can choose the "custom" option and create the internals for Virtual Dub job script yourself. You just need to leave out the script calls for VirtualDub .Open, .subset, SaveAVI and Close. My script will add those automatically with the correct frame settings and file names. The final option (second tab of the dialog) is the list of media clips found on the selected video track. Check the checkbox next to those that you want to run the script against. I strongly recommend just one track and one video clip for now though.. It shouldn't screw anything up otherwise, but hey, this is kind of a beta here.. So here is the script (my first one and a bit of overkill personally): http://www.szudzik.net/vegas/scripts/24pWorkflow.html Click save and place it in your vegas scripting folder. I'm still going to work on this some more. Right now, it will process the whole file represented by each event on the timeline. It ignores clips, markers and all of that. I just grab the VideoEvent and get the underlying media file path and will operate against the entire file. As a result, it's best to just test this with a single track and a single video on the track. The script dialog also allows you to load and save "template" scripts that you can use to update the script dialog for how it operates. This doesn't apply to the list of clips, but only to the general settings tab. The "Clean" button on the dialog will DELETE ALL files in the working directory specified on the dialog. Make sure that's what you really want. I'm not responsible if you delete anything accidentally! Really, I'm not! The Close button works as a "cancel" or "done", click it when you want the dialog to go away. The "Run" button will start the workflow process. Depending on what options you choose and how big your file is, it might take a few seconds or significantly longer. Each of the resulting AVI's generated by VirtualDub will be placed in the working directory with the same name as the original video file, but with _24p.avi appended to the end. I think that pretty much covers it for anyone who wants to give this a try. Like I said, try 1 clip first and see if it does what you want. I'm going to get a page crafted with the above info and some screen shots to serve as my "Help" file, hopefully done in a little while with that. I'm going to comment the source code and will later make that available as well for anyone who wants to make changes or improve the workflow. Again, this is "BETA" type software here, so it's a use at your own risk type of thing. I'll fix up any issues you guy's find and listen to suggestions for it as well though. --Steve Last edited by Steve Szudzik; May 16th, 2007 at 08:14 PM. Reason: a few small typo's

May 17th, 2007, 09:16 PM	#4
Steve Szudzik Regular Crew Join Date: Mar 2007 Location: Sammamish, WA Posts: 103	Thanks for the feedback Mike. Yeah, I just threw those instructions together quickly. I'll get them updated soon hopefully. I today also got the dialog from VDub about the cineform codec, guess my trial has ended ; ). It still won't uninstall but interestingly, I don't have the logo either. Maybe a bug with vista and how they uninstalled. For everyone else. I have also made this "script" into a stand alone executable that you can use to automate and batch the workflow. You can find it at the below location: http://www.szudzik.net/vegas/scripts/HV20Pulldown.html It's C#, so requires the .Net Framework 2.0, just like vegas but his doesn't have any Vegas dependancies. The UI is nearly identical to the vegas script, so this page should look familiar. The difference here is that one the 2nd tab, you can select all the media files, from wherever you want and add them for processing. This version also does not require the copying of the file to the "working directory" so it's much faster. Let me know what y'all think.. Steve Last edited by Steve Szudzik; May 17th, 2007 at 09:49 PM. Reason: changed exe link to documentation link.

May 19th, 2007, 02:04 PM	#7
Mike Dulay Major Player Join Date: Mar 2007 Location: Somerville, NJ Posts: 304	Working like a champ Steve, your standalone exe is working great. I've got a harddrive full of m2ts that I'm converting right now. Wish my new laptop was faster but hey, I'll take what I can get. I'll have to make a new blog referencing how to put your exe together with all the other freeware software. Can't have you doing all the work. 8-) Thanks for doing this! __________________ DIY, 35mm, HV20: http://www.primitivebuteffective.net

May 19th, 2007, 06:53 PM	#9
Ajit Bikram Regular Crew Join Date: Apr 2007 Location: Baltimore, MD Posts: 71	What I might be doing wrong? I downloaded all the scripts and the self-executable file. Also had to install xvid. The program works and gives an output. But the output definitely doesnot look to be progressive!! (While playing I can see the combing/ghosting). To test I did the following to compare.... 1) Converted the m2t file to avi. 2) Opened the avi in VirtualDub. 3) Used the telecide filter 4) Played the output-- this definitely does not have those artifacts and is progressive. Looked at it frame by frame. But it does have one duplicate frame (every 5th) as it should. So what I might be doing wrong???

May 19th, 2007, 07:35 PM	#10
Mike Dulay Major Player Join Date: Mar 2007 Location: Somerville, NJ Posts: 304	Odd ... Ajit, are you manually doing the process rather than letting Steve's exe create the avi for you? The exe should have done the TFM and TDecimate for you already. #3 on your list should not be necessary as there is no longer any pulldown for you to process with telecide. __________________ DIY, 35mm, HV20: http://www.primitivebuteffective.net

May 16th, 2007, 08:56 PM	#2
Steve Szudzik Regular Crew Join Date: Mar 2007 Location: Sammamish, WA Posts: 103	Ok, I found one small bug (new I rushed it a touch) and have posted a quick updated DLL on the page. The bug caused the process to fail the first time through because it didn't detect the AVS generated by DGIndex. The second time through worked properly. It now works the first time as well as subsequent times. --Steve

May 17th, 2007, 08:39 PM	#3
Mike Dulay Major Player Join Date: Mar 2007 Location: Somerville, NJ Posts: 304	Some notes Steve, you may want to update your instructions on how to invoke the script dll file. I was looking around Program Files when I realized you could store it anywhere. Ran into a roadblock with the Cineform HDLow default compression in the script. How I ran: 1) Downloaded Vegas trial from sony 2) Installed and registered trial version 3) Dragged and dropped an m2t file unto the Vegas timeline (setup for HD size with 23.796fps) 4) Stored your szudzik.HV20Pulldown.dll unto c:\clip (arbitrary choice -- probably should be C:\Program Files\Sony\Shared Plug-Ins) 5) Invoked Tools -> Scripting -> Run Script... -> specify location of the dll 6) Specified c:\temp as my work directory 7) Specified C:\clip\24p\dgindex\DGIndex.exe as my dgindex path 8) Specified C:\clip\24p\VirtualDubMod_1_5_10_2_All_inclusive\VirtualDubMod.exe as my virtualdub path 9) Specified C:\clip\farnsworth.avs (this Farnsworth's original template with -222ms ... from here ... http://www.hv20.com/showpost.php?p=1421&postcount=10) 9) Selected Calculate delay audio 10) Left the default option CineformLowHD 11) Run DGIndex runs okay, generates all the files in c:\temp, syl file is generated and virtualdubmod 1.5 launches. Now I get the immediate message "License Required: This application is not licensed for CineForm Hd encoding. Please visit www.CineForm.com to purchase your upgrade." I click okay and virtualdubmod continues to run. The generated avi file in c:\temp displays the cineform logo (too bad, the file looked good and was only 1x larger vs 33x). When this is dragged on to the vegas timeline you can see that the logo is hardcoded into the avi. I think you should rather default to Uncompressed (but recommend in the dialog LagarithLossless or HuffYUV codec). I repeated 10-11 but with Lagarith this time. 12) I add back the 24p AVI file to the timeline ... rendered to HDV intermediate. I can probably use this as my archival format and edit format (but not as a playback) ... for final render I would use a common codec like AVC/H264 or WMV. One thing I can comment, it is so clean from a use perspective! I think you'll help sell more copies of Vegas. 8-) __________________ DIY, 35mm, HV20: http://www.primitivebuteffective.net Last edited by Mike Dulay; May 17th, 2007 at 08:49 PM. Reason: clarifications

May 18th, 2007, 09:54 AM	#5
Ajit Bikram Regular Crew Join Date: Apr 2007 Location: Baltimore, MD Posts: 71	Downloaded your exe file. Will report how it works once I get some 24p footage this weekend! ;-) Thanks for all your effort and time!

May 18th, 2007, 10:06 AM	#6
Steve Szudzik Regular Crew Join Date: Mar 2007 Location: Sammamish, WA Posts: 103	No problem, hopefully it will work well for you. I might make a few more updates to the standalone app this weekend. I might "hide" the DGIndex window. It's such a short process, so you don't gain much by watching it work. I wouldn't mind hiding the VDub window as well, but since I can't get any progress info back from it, I'm reluctant to do that. At least with that window up, you can see what's going on. I think I prefer the standalone app right now myself. The Vegas one can be good, but I'd have to put in a lot more work. It would be nice if it honored the splits and markers etc.. Someday maybe ; ) --Steve

May 19th, 2007, 02:36 PM	#8
Steve Szudzik Regular Crew Join Date: Mar 2007 Location: Sammamish, WA Posts: 103	Thanks Mike, glad that it's working well for you. I just updated my documentation for the application at: http://www.szudzik.net/vegas/scripts/HV20Pulldown.html Put in the links to the required apps as well as a better workflow on how to get started. I also made some small changes to the app last night. Got rid of the "clean, run, load, save" etc.. buttons and put them in a file menu. Looks a bit cleaner. Also doc'ed most of the code and should be posting that up shortly. Funny thing is there's more code for working with the UI than there is just to run the workflow! Could easily be fully scripted in a batch file as well, except then it's tougher to select the video files, unless you just batch process an entire directory. --Steve

May 19th, 2007, 07:41 PM	#11
Ajit Bikram Regular Crew Join Date: Apr 2007 Location: Baltimore, MD Posts: 71	Hi Mike, I tried doing it using Steve's exe. But that did not work out! That's what was my point saying that the output is not progressive! However when I used the filters manually in virtualDub it gave me progressive. can you think of any possible reason?

May 19th, 2007, 07:44 PM	#12
Mike Dulay Major Player Join Date: Mar 2007 Location: Somerville, NJ Posts: 304	Ajit, Steve's exe isn't standalone. You probably did not install the TIVTC plugin into avisynth. Look for post #26 on this thread ... http://www.dvinfo.net/conf/showthrea...t=91500&page=2 __________________ DIY, 35mm, HV20: http://www.primitivebuteffective.net

May 19th, 2007, 08:26 PM	#14
Steve Szudzik Regular Crew Join Date: Mar 2007 Location: Sammamish, WA Posts: 103	If the specific VirtualDub script that you selected in my app (I call them "compressor") doesn't quite do what your looking for, you can use VDub to get the settings how you'd like. Do that and the click "Save AVI" but instead of saving, tell it to create a job. You can then look at the jobs in VDub and save those to a file. Open the job file and you can then bring that in as a "custom" script on my UI. Just leave out the Open, Close, SaveAVI and .subset portions of the script. It could also be something in the .AVS template file as well. I know that Farnsworth & Mike both have a few different settings between them. Anything you can achieve by just using DGInex & VDub can be done in my app. I'll I'm doing is automating the files through those two applications. --Steve

May 19th, 2007, 09:32 PM	#15
Ajit Bikram Regular Crew Join Date: Apr 2007 Location: Baltimore, MD Posts: 71	Can you tell me when I install the TIVTC plug-in, I just have to copy the TIVTC.dll file into the avisynth plug-in folder. Am I correct?