Creating those multi-stream videos from phones

[Paul R Johnson]

I've finally completed one of those multi-source videos for a choir. You know, those fund raisers for charities where everyone sings at home, and you edit it into a finished product. I never assumed it would be easy, but wow - what a time consuming thing to do.

Plenty of anticipated problems, but so many caused by technology.

Video via Adobe Premiere, and the audio via Cubase.

The plan was to make guide tracks for them to sing to, and put these on youtube. Sing to them with headphones in and record on the phone.

Initial proress fast - receive an email, log into online shared storage, find the right track - difficult when their labelling doesn't help, and download it. Discover weird file formats - like the Android ones, convert them to something sensible and stick it into Premiere.

Export the audio to Cubase, on a different computer - BIG time waster with files going back and forth.

Create the choir in Cubase usually while waiting for Premiere to render. Something that got more and more annoying as the track count rose.

The final edit looked OK until I noticed the sync had gone wrong - and some tracks were playing at random speeds, when they didn't originall. Eventually I found 20 versions previously a morph to cover a jump cut where everyone missed a tempo change in the audio track had gone wrong. At the time it didn't matter, but as track count went up, at first renders suddenly took twice the time, then because unstable, then crashed continually. Going back, and redoing everything from that point cured it, but I was tearing my hair out.

If anyone fancies doing one, I'm now full of tips - so just ask and I can maybe help you not waste so much time and effort.

I also now know how 40 sources with shrinking and screen placement, masks and crops really tax the system.

YouTube (Short URL)

[Patrick Baldwin]

Thanks for sharing Paul. I am amazed that it sounds so good given the sources.

[Paul R Johnson]

Cubase has some very useful features to assist and these work well on solo singers.

The bigest problem, which I discovered when our band switched to IEM monitors is that with both ears plugged up it is VERY easy to be consistently sharp or flat and with 38 tracks the way phrases are sun and the pitch accuracy need some assistance. They all sing to a track, so you might get these variations:
Lovin'-dat-man-of-mine
Lovin'dat -man-of-mine
Lovin'-dat-manof-mine
Lovin'-dat-man-ofmine

or other versions - which need sorting out so they align. Cubase actually does auto alignment, but it was ineffective her, and mean slicing each track up and moving it forwards or backwards. Individually, they were fine, but it was a mess together.

[Bryan Quarrie]

Very well done, Paul.

Regarding the clips themselves, did you import them on a single drive, or did you spread them across multiple hard drives for editing?

(I am doing something similar for a local community group. Each member sends me their video via Dropbox, and I convert them to a common resolution, framerate and MP4 format (yes, I can't stand those weird Android video formats either 😑).)

[Paul R Johnson]

All on the same drive, and resizing done in Premiere. The odd Android strange one was the only one that needed conversion, other than that the files were straight from google drive and drop box - not everyone seemed able to cope with the file transfers.

[Colin McDonald]

Well done, Paul. Great job.

[Paul R Johnson]

I'm doing a different one - Vivaldi's Gloria with an authentic Baroque quartet and organ, and 38 singers. This is taxing me in different ways. The biggest issue here is that without them being in the same room, I've got lots of flexible timing. I gave them a guide track, with one person singing a guide track - it's conventional SATB. I also gave them a pretty loud click. The words are pretty much Gloria, in Excelsis Deo. The problem is that in the Baroque period, the choir would sing Glor-rya, the last syllable being sung quickly with the glory extended, and the excess pronounced ec-celsis, and the choir are mangling both. Baroque rhythm tends to do this weird extending one bit and shortening the next, more than the actual score suggests - double dotted rather than the single in the scores. With nearly 40 singers, this is proving somewhat tricky - lots of chopping up and slipping and sliding. The quartet and organ sound quite nice, apart from the horrible clacking from the organ manuals.

Not sure yet what to do with the pictures? Perhaps thinking of something softer than rectangles?

38 singers, the conductor and 4 musicians?

Any suggestions or crazy ideas for a period piece?

[Pete Cofrancesco]

I never mentioned it but I enjoyed you first effort.
Note with the numbers you won't have enough room. Your last one you 4 rows x 10 columns with the conductor taking 2 spots for a total of 39 ppl.
That wouldn't leave you with enough spots for 38 singer and the conductor taking only one spot (not wide enough). You could leave off the musicians or fade them in and out over 6 of the sings.

As far as doing something different, I'd stick with the same grid but, if there are soloists you could enlarge them for their solos much like a zoom meeting, and you could mix in images that sweep across, if that's not too distracting. These images could relate to acts of kindness and community during the pandemic or just visual effects, although it might be a distraction if not executed well. But I think it would be nice if you could mix in some sort of slideshow. To provide more visual interest and depth...

As far as the technical timing challenges I don't think I can be much help.

[Paul R Johnson]

Not thought about the images moving, but that could be good. The trouble with this piece is that the singers come in about 20 bars in - so the first bit is just the conductor, two violins, viola and cello. I also have a snag in that the viola player seems to have gone AWOL, so I have her audio track, but no visual. The 1st Violin says he could video himself playing the viola - but then I'll have two of the same person????

I wonder if I could dump the squares and put in blurred oval masks? Or position them like a real choir - conductor centre bottom, with the strings grouped around them, then semicircles of singers? Would that look a bit messy? The pictures would also be quite small with so many people?


Everyone sings the same apart from two bars where two groups start a line, and the other two join in, so maybe keeping them in lines according to section would work?

[Pete Cofrancesco]

I would keep the grid and just play with zoom into soloist or mixing in photos or video of other things. At least that's my take. I find these things always take more work than you anticipate and then you're in a rush to get it done by the deadline and all your fancy ideas go out the window.

[Paul R Johnson]

The time and effort for see is immense and not reflected by the 'price for he job'. Luckily time isn't too short at the moment but no way could you do these as part of normal video work. I've not even started edit properly yet and to get to that stage is probably 30 hours work. I think this will be my last. Worst is that hanging around time is crazy. Forty plus timelines, each with shrinking and masking makes render time massive. I've broken the project down to six sub-sequences, so I can do each section's tweaking separately, then combine them. I wish premiere had some kind of rulers, or lines that stay in place between sequences.

I'll report back.

[Pete Cofrancesco]

I recently did a similar project and I made grid like mask in photoshop that was imported into my editor and placed on top. Kept everything clean and consistent. Luckily I didn’t have 40 channels to deal with.