What "free" 24p workflow for Windows users?

[Mike Dulay]

code and proposed mods to farnsworth's:
The code below is for fixavsdelay.pl. You need to install ActivePERL 5.8 which is free. Save it to the same directory as go.bat. In go.bat you can create the template.avs as noted in the comment lines. Also modify go.bat with the comment lines below. Once it runs, two avs files are generated. One is the original made by dgindex. The other has a similar name but ends with _fixedaudio.avs which adds the delay value from the filename. It's messy yes, I didn't want to overwrite the original AVS incase you don't like how this works.



#!c:\perl\bin\perl.exe
# This file is fixavsdelay.pl
# You can install ActivePERL 5.8 to parse this for you
# Parse AVS files in directory with proper delay

# My template.AVS:
#vid=MPEG2Source("__vid__")
#aud=MPASource("__aud__").delayaudio()
#audiodub(vid,aud)
#TFM(d2v="__vid__")
#tdecimate()

# My go.bat
#COPY template.avs %1
#FOR %%f IN (%1\*.m2t) DO dgindex\dgindex.exe -if=[%%f] -FO=0 -OF=[%%f_] -AT=[%1\template.avs] -EXIT
#FOR %%f IN (%1\*.m2t) DO perl fixavsdelay.pl %%f


sub mstos {
my $ms = $_[0];
#preserve the "-" sign if present
my $sign = "";
$sign = '-' if ($ms=~/^\-/);
$ms =~ s/^\-//;
my $secs = sprintf("%.3f", ($ms/1000));
return $sign.$secs;
}

my $filename = $ARGV[0];
open(AVSFILE,"<$filename\_\.avs");
open(OUTFILE,">$filename\_fixedaudio\.avs");

foreach my $line (<AVSFILE>) {
if ($line =~ /DELAY /) {
my $subline = $'; # part of string after word DELAY
$subline =~ /ms\.mpa/; # part of string between delay and ms.mpa
my $delaystring = $`;
my $seconds=mstos($delaystring);
$line =~ s/delayaudio\(\)/delayaudio\($seconds\)/;
print OUTFILE $line;
} else {
print OUTFILE $line;
}
}
close(AVSFILE);
close(OUTFILE);

__END__

[Ajit Bikram]

Originally Posted by Mike Dulay

code and proposed mods to farnsworth's:
The code below is for fixavsdelay.pl. You need to install ActivePERL 5.8 which is free. Save it to the same directory as go.bat. ..........
_

This is just incredible Mike! I would try to give this a spin this weekend!
I wish I would have learned perl for my work too!
Thanks a lot!

[Steve Szudzik]

Perl... **shudder**

Gave this a try and it does seem to work fairly well. I was looking to maybe create a script for Vegas based on what Farnsworth had come up with. The only big difference between his audio delay and yours was .222 vs. .237. Very small indeed.

Between this and Farnsworths calls, I think we might have a winner until NEO is supported on Vista (and my wife stops her coughing/seizure fits when I tell her I need another $300 software package to edit our video). I'm always willing to Beta test as well btw. At work, just about everything I use is pre-release, so I'm used to filing rich, detailed bug reports and even doing some debugging (C#, C++ on occasion).

--Steve

[Mike Dulay]

Steve,

Farnsworth and mine have different delays because we're running different clips. Actually, every clip you extract will have a different delay. That's the reason for the PERL to read what delay value is right for each file. We can polish this some more as we go along.

UPDATE:
The script files can be had at http://www.sendspace.com/file/71jk3n
The detail of this procedure are documented at http://yousillyman.blogspot.com/2007...-hv20-m2t.html

[Mike Dulay]

Sample video produced after IVTC:

91MB WMV9 720p
http://www.sendspace.com/file/lu6m1q

[Steve Szudzik]

I'm working on a Vegas 7.0 Script DLL that will fully automate the process as created by Farnsworth. The first version is going to be basic as all get out. Basically you'll put 1 24p clip into the timeline and then run the script. The script will check for the DGIndex location, template file and VirtualDubMod paths. After confirming, it will automate DGIndex & VirtualDubMod to produce the AVI. I've got that 100% working right now.

My next step is to alter the vegas timeline (it'll be an option) to 24p and load up the created AVI file from VDub. Also optionally, it will automatically render out the file to M2T since the AVI is just bloody huge!

I'm thinking a later update will allow you to have multiple clips, should be easy.

If you used this script, would you have issues with it writing to the HKCU registry to preserve the template, DGIndex & VirtualDub locations? ALso, the script copies the original file to a temp directory in users\<name>\AppData\Local\<myAppName>\ and processes them in that folder. Would people prefer to specify a processing folder, or even utilize the original file location? I don't like the original file location for processing because of all the other files that DGIndex puts there. I can see the final AVI or M2T being placed back in the original location though (rename the M2T to _24p.M2T or something).

Also, do folks have problems with a DLL script vs. a regulard C# script? I'm always more than happy to post the code.

Any ideas of a desired flow for this would be greatly welcomed. Time permitting, I might have something for folks to "beta" test late tomorrow (got to do real work at some point :) ).

- Steve

[Mike Dulay]

Steve,

This sounds great. Doesn't Vegas use an embedded cineform for generating m2t? If so, it would be desirable to do it that way rather than generate an AVI, as you said they're huge.

Writing to the registry might be tricky for those of us that have locked down PCs ... why not a temp file? As for the destination for intermediate clip, I'd prefer a configurable location as space on the C drive may be at a premium during large edits.

Does scripting come in the trial version of Vegas? Didn't know this was possible. I'd like to help as time permits (~not very good with compiled languages).

[Steve Szudzik]

Yes Vegas will create a m2t, but VirtualDub (which does the avisynth script) will create an AVI file. After I get the AVI from VirtualDub, then i'll put it on the timeline to render out the M2T.

A temp file would be ok, in the user app data folder. HKCU is just normally where I'd put stuff like this, but a data file is fine as well.

The Vegas trial version should support scripting. It's fully functional, just times out after 30 days.

I was trying to script up the delay like you did in Perl, but was having some issues. (I really don't read Perl but made a stab in the dark) For example from my AVS file generated by DGIndex:

C:\Users\steve\AppData\Local\Szudzik.HV20Pulldown\Short_ MPA PID 814 DELAY 0ms.mpa

You seem to grab what's between 'DELAY ' and 'ms.mps' to get the delay value right? In one sample of mine it winds up being 0, which seems odd. I'm still playing with that, but wanted to double check and make sure that's essentially what you were doing..

[Gavin Ouckama]

Mike,

What would be really cool is if you could comment line by line what you are doing. That way, for those not familiar with PERL could perhaps give a stab of the same thing in other languages. In addition, it would, at least for me, just be nice to know exactly what each line of code is doing.

All involved in this thread rock as far as deriving a "free" solution here :)

Thanks in advance ...

[Steve Szudzik]

I made pretty good progress last night with my Vegas script. Managed to get everything from the timeline through the final 24p AVI full automated, no user intervention required.

Tonight I'm going to try to get it back on the new 24p timeline and render the final 24p M2T file. Was working on that last night but having problems getting Vegas to create a new project. It was way too late at night to be coding. Apparently Vegas doesn't like to create new projects or tweak video settings on a background thread. Not too surprising. Just means that I'll need to slightly change how I was doing a few things. Otherwise the script (my first for Vegas) is progressing nicely.

Gavin: I'll make sure I fully comment the code as well ; )

Steve

[Mike Dulay]

Steve,

You are correct. In PERL, the =~ is the pattern matching operative with syntax like grep (uses regexp). As for the 0 you are getting, you are probably using integer operation without decimals. I'm reposting the code with full comments for those that want to convert it.

code process
1) go.bat runs a loop and feeds fixavsdelay.pl the name of each m2t file
2) fixavsdelay.pl derives name of avs file based on DGIndex name convention
3) A new _fixedaudio.avs file is created
4) fixavsdelay.pl loops through the original avs file seeking the line with work DELAY, the rest pass through to _fixedaudio.avs
5) if line with DELAY is found, get the millesecond value and convert to seconds, change the delayaudio() parameter with a seconds value with 3 decimal places

sub mstos {			# declare subroutine
	my $ms = $_[0];		# accept subroutine parameter, a millisecond delay value
	#preserve the "-" sign if present
	my $sign = "";		# declare default sign value as blank
	$sign = '-' if ($ms=~/^\-/);	#if $ms begins with '-' then save that $sign as '-'
	$ms =~ s/^\-//;		# clear the sign value from $ms so we can do good math
	my $secs = sprintf("%.3f", ($ms/1000));	# convert $ms to $sec with three decimal places of floating point math
	return $sign.$secs;	#return value of $sign and $secs,  e.g. 0, 234, -234
}

# This is the main program
my $filename = $ARGV[0];   # Accept 1st parameter to PERL as $filename
open(AVSFILE,"<$filename\_\.avs");  # open filename_.avs for reading, the one DGIndex generated
open(OUTFILE,">$filename\_fixedaudio\.avs"); # create our own filename_fixedaudio.avs for writing

foreach my $line (<AVSFILE>) {		# read the DGIndex AVS file line by line
	if ($line =~ /DELAY /) {	# if you find the word DELAY
		my $subline = $';	# get part of string after word DELAY
		$subline =~ /ms\.mpa/;	# then part of string between delay and ms.mpa
		my $delaystring = $`; 	# this leaves just the value in between, eg. -237
		my $seconds=mstos($delaystring);	#convert ms to sec for delayaudio
		$line =~ s/delayaudio\(\)/delayaudio\($seconds\)/;	#replace string delayaudio() with delayaudio(value)
		print OUTFILE $line;	# print the replacement to our output file
	} else {
		print OUTFILE $line;	# all other lines not containing word delay simply pass through
	}
}
close(AVSFILE);
close(OUTFILE);

__END__

[Steve Szudzik]

I think for my couple of clips, the zero value was actually ok. After going through and rendering them out as 24p WMV, the audio was nicely in sync. Both of those clips that produced a 0ms were very short clips. One longer one I had specified a 8ms delay. I haven't had one request anything larger than that yet or anything with a negative delay, but again all of my test 24p clips are very short, under 30 seconds.

--Steve

[Mike Dulay]

Steve,

Fixed the messy formatting from the boards software. My delays range from -240ms to 4ms. More of them are '-' value. My clips range from a few seconds to several minutes. For the conversion, look to the sprintf line .... incidentally that command is derived from C so C# / JS should have a similar construct.

At -200ms you'll see a full second lip movement to sound deviation in a >5 second clip. So you've been lucky with a 0-4ms deviation.

[Steve Szudzik]

Once I get the workflow more solid, I'll start playing with some larger files. It just takes a while for DGIndex & VDub to do their things, so I want to keep the testing cycle short until I'm fairly certain it's working properly.

There's a few ways to format the proper millisecond value in C#. A quick and dirty (that's the kind of programmer I am ;) would be just to let some of the built in classes do my work for me like:

// msValue would have the string representation of the milliseconds
// as pulled from the AVS file (just hard coded a value here).
string msValue = "-237";

// Finally, convert the string MS to an integer using System.Convert and pass
// that into the TimeSpan constructor as the total milliseconds.
// The .TotalSeconds property will convert and return our fractional seconds
// to 3 decimals which we cast back into a string to update the msValue.
// property
msValue = new System.TimeSpan(0, 0, 0, 0, System.Convert.ToInt32(msValue)).TotalSeconds.ToString();

-- Steve

[Mike Dulay]

Steve,

Cool, so you've figured out how to parse out the millisecond value using C#? Can't wait to try the Vegas way.



An aside:
How are you processing the vdub portion of your workflow? Batch mode in full processing or fast recompress? Uncompressed RGB or lossless codec as an intermediate? And how fast does it run for you?

Sorry, that's a lot of questions. My PC is a Core2 1.66 and it does 1.6-2.0fps in the vdubmod phase using MSU/WMV9 in fast recompress. Lagrith reaches 3fps and up (this is the preferred for an intermed. lossless -- http://compression.ru/video/codec_co..._2007_en.html). The difference between full and fast is supposed to be small in processing time but full does a colorspace change as well. Visually I see the colors get more saturated ... maybe darker by a bit with full processing ... but it could be a mind trick. I prefer to run with fast recompress. Just things to think about if the process needs optimization to make it quicker later on. Also noticed that vdubmod doesn't seem to take advantage of a multi-cpu setup as I can even watch videos while vdub is running.