Syncing live performance to studio performance

[Rob Perry]

I'm producing a promo video for a bands' upcoming tour and have been given live video of them to work with. While the video looks great, the audio is atrocious and beyond useable. My initial reaction was to try and sync the live performance with a studio recording of the same song, crossing my fingers that the band might have impeccable/consistent timing. After 30 minutes of playing around with it in Premiere (adjusting the playback speeds, etc.) and realizing that the live audio's timing fluctuates too much to sync with, I'm wondering if there might be some kind of software that can dynamically match the two audio sources... like some sort of temporal quantizing?

I realize this is a ridiculous request, but after sitting through a bunch of seminars at NAB this week (specifically on working with audio in the Asobe CS environment - Premiere/Audition) I feel like there might be something out there that can do this. Am I completely naive to believe this?

[Battle Vaughan]

Taking a stab at this, as I have not used this myself, but have heard good things:

Red Giant - Products - PluralEyes 3.1.1 - Features

For mac, a Windows version should be available very soon. Apparently a Windows version of an earlier iteration of the program is included, however. Free trial.

[Richard Crowley]

This is why the typical method is to "lip sync" video "performances" to the produced studio recording. Probably 99.99999999999987% of all music videos are made this way, and you have proven first-hand why they do it that way.

Since you are not trying to sync the SAME sound, Plural-Eyes seems rather unlikely to be of any help here.

[James E. Thomas]

I just finished a project for a CD Release Party in Albuquerque NM and was faces with the same problem. I could not get a feed from the soundboard and they were going to record the concert to cd which also failed for some reason. Like your situation the audio from my cameras was horrible. Ten out of fourteen songs were performed to studio recorded soundtracks which were not very dificult to lip sync reasonally well. The four songs they sang using live instruments were a huge challenge but I finally nailed them pretty well although it took me about a week off and on to get it done. I had to cut the audio tracks in several sections where needed and adjust the audio speed and even some speed adjustments on the video. It took a lot of patients and tedious work, Good luck!!

[Roger Gunkel]

I have done this sort of synching a few times over recent years and my advice would be to synch the live video to the studio audio, not the other way round. Variations in the speed of the audio are much more likely to be noticeable than in the video.

If there are different camera angles in the shoot, then it should be pretty straight forward to cut in a non timing specific shot such as a facial close up of a non singing member, then you can reset the video synch for the next on beat shot. Repeating cutaways in a music video, especially instruments, will never be noticed over the length of a song and can easily leave space for timing resets. Slight variations can always be adjusted by changing the video speed of a clip or even turning a shot into slomo.

Roger

[Paul R Johnson]

If you import the video into an audio sequencer, like Cubase, it's fairly simple but time consuming to time stretch the audio to fit the video. However, the real snag is that lip sync and drumsticks are going to be a real problem, because matching the drums AND the vocals will be very tricky, because very rarely will the timing work. If they have supplied you with something cut from multiple cameras, then if you can get the original material it will work better, but if you have just one camera your problems are just beginning. You can approach it by cutting up the video and slipping the drum shots to match the drums and the vocals to match the singer's lips, but you end up with lots of gaps, so you need material to cover. Music videos are never easy because musicians will be amongst the viewers and will spot these musical errors very quickly.

[Warren Kawamoto]

Hopefully, you have many video angles to work with. If you do, you simply use the soundtrack as a master bed, then lay quick video shots from different angles over it.

If you only have one continuous video to work with, you're in for a very long and time consuming editing session. You'll have to lay the sound first, then variably adjust the speed of the video to match. Playing with the speed of the audio in this case won't work; the musicians will be able to sense that the timing is not right because some parts will be off the beat.

[Pete Bauer]

Rob, I don't know how well it would actually work for your particular situation, but Adobe Audition CS6 has a feature called "Automatic Speech Alignment." Could give it a try:

How to use Auto Speech Alignment in Audition | Digital Video CS6 | Adobe TV

http://helpx.adobe.com/pdf/audition_reference.pdf

[Gary Nattrass]

Just edit the video to the studio track and with skill and quick cutting you will get a useable promo, Ive done loads where live footage has been added to a studio track and normally it will hold sync for a couple of secs which is more than enough for most music.

Ive also done videos where mutiple nights have been used as a source for concert footage!

[Richard Crowley]

Originally Posted by Gary Nattrass

Just edit the video to the studio track

Absolutely!

Cutting or munging the audio to match the video will ALMOST NEVER yield acceptable results. I would categorically reject that method altogether. (As thousands of other editors have done before us.)

[Trevor Dennis]

If he is using Premiere Pro, then I think I'd be inclined to cut the audio track into a number of manageable Sequences, Lock the audio track, and place each video clip on alternating tracks 1,2,1,2 etc. Then use time remapping for each video clip in the effects panel. Set the 'toggle animation' clock in the effects panel so you can adjust the clip speed dynamically.

When the cuts no longer meet, as is bound to happen, having the clips on alternate tracks will make fine tuning the cut point super easy. Or you could overlap the clips if you want to use video transitions.

Then put the sequences back together in a master sequence. They will match perfectly.

[Battle Vaughan]

Originally Posted by Richard Crowley

Since you are not trying to sync the SAME sound, Plural-Eyes seems rather unlikely to be of any help here.

I quite defer to Richard on this, but would point out that the 3.0+ versions of Pluralize have a specific bin for prerecorded sound, and in fact it seems popular in making music videos synching to a pre-recorded track. Here's a tutorial, skip the early blather in it...

Pluraleyes 3 Review on Vimeo

Point well taken, however, in editing the video to the audio rather than vice-versa. Again, I have not personally used this program and I am not promoting it, but it seems a useful tool which appears to do more than simply link audio in multi-cam environments.