How to (and How Not to) Edit a 38-Camera Project: Part II

Guest post from Jacob Stone, Founder & Creative Director of Punch Drunk

THE EDIT: ROUND 1 (THE UGLY)

A lot of what I do is video for live music: directing concerts live at music festivals and reacting quickly to what’s happening on stage. So I figured this would be a great approach to cutting the concert: essentially recreate a live scenario in post-production. We actually did some tests to make all 38 cameras available at once and edit that way in Premiere (including some crazy keyboard remapping). No matter what we did, there were some inescapable lags in the system, so instead we decided to approach the edit by city.

The thinking was: I will cut a multicam edit from each city, and then at any given moment in time, the best shot from each city will be available for a “meta” multicam project where each edited city is one “camera.” This seemed like a pretty solid plan, since theoretically you would only ever view the best possible shot, so cutting those together would be a breeze. It would look something like this:

So I gave that a whirl, editing five 90-minute concerts for 7.5 hours over several days. Then I began assembling the “meta” final video. After three minutes I realized this would not work.

What I failed to consider is the content: music. Like many live directors, I cut to the beat of the music. It’s natural and it makes sense. The problem in this case is that I had cut to the beat of the music five times, and then combine those edits together, resulting in “stepped on” edits, where I instinctively made an edit during the “meta” edit at almost the same time I did previously – so you’d have an edit for 1-2 frames and then it would switch.

It looked terrible. Back to the drawing board:

THE EDIT: ROUND 2 (THE BAD)

Since everything was already setup for editing by city, I figured this process could still work, but I would have to rethink my approach for editing individual cities. Instead of cutting to the beat, I would have to cut OFF the beat, choosing the best LOOKING footage instead of timing my cuts to the music.

So I sat down for another 7.5 hours. This time, instead of turning the volume up and rocking out (as I would at a live show), I put in earplugs so I could better focus on what I was seeing, without eliminating the music completely.

Once I had my new and improved city-by-city edits complete, I attempted once again to make the final product. And again, after three minutes it was clear this would not work.

This time the issue wasn’t so much timing as content. Suddenly I was paying more attention to what the viewer SHOULD see at a given moment, such as a guitar solo or drum fill, but because I had gone through each city and cherry-picked the best shots by city, I effectively hid a ton of the relevant shots from myself.

For example, I may have liked a shot from Pomona of a killer guitar solo, and cut to that for 30 seconds or so in that city edit. But then in the final edit I didn’t have access to any of the other city-specific shots, like an aerial view of the drummer during that guitar solo.

At this point I realize that cutting everything by city simply will not work. Since the camera angles at each city were determined by available lighting and mounting options (not systematically by performer), cutting each city as it’s own sub-edit just won’t work. It’s back to the drawing board again:

THE EDIT: ROUND 3 (THE GOOD)

After 14 hours of multicam edits, I’m ready to try anything that will speed up the process. So instead of cutting by city, I decide to build the sub-edits based on how the music relates to what the viewer expects to see. The vocals are the thread holding everything together, so I choose them the “foundational” layer. The guitars drive the melody, so pass number two will be guitars. The third pass is drums, since you often need to inject drums quickly in-between other shots. This means the edit will be structured with all the necessary singer and guitar shots, and then drums and crowd shots can get peppered on top to tie it all together.

After this much testing and burning-in of the editing system, I’ve determined that it handles nine simultaneous HD layers in real-time as a multicam project pretty well. Good enough for a full-length live cut for this project. And since nine makes a nice multicam matrix AND means you can assign each multicam angle to a button on a numeric keypad, this seems like a solid approach.

So I do these edits. They work and look good. I complete the first pass on the singers, then use that as “camera 1” for the guitars multicam. This means “camera 1” is actually a sub-edit of all the vocals. I can layer guitars on top of that from the eight remaining shots. And that works.

Then I insert the guitar multicam (which contains the singers multicam) as “camera 1” into a third multicam, which contains eight shots of drums and crowd shots. This seems to work. Until it comes crashing down.

Towards the end of completing the third pass, I start to encounter some strange behavior in Premiere. It starts running slowly. It bogs. It crashes. And suddenly it won’t open my underlying first and second pass from my third pass project. W. T. F.

I spent weeks gently massaging the project file back to life, extracting sub-edits (first and second pass) into new projects, relinking files, and carefully rebuilding my work. Finally, after pulling out a lot of hair (and I think literally having some hair turn gray), I finish the third pass and everything seems about right. Unfortunately I had to “hard burn” the first two passes into ProRess 422 files of their own, thus losing the ability to go back and tweak those edits. I’m able to salvage the work and insert them into the third pass though. I make all the necessary tweaks to have a “first final cut” of the project.

Proud of my hard work and weeks of frustration, I happily export a full version and upload to Vimeo for review. Sascha calls a day later. “Jacob, the video looks great, but I have to say – it’s out of sync.” We go back and watch together. I notice a couple of blips and bumps, but overall it looks great to me.

“I’m sorry man, but there are entire sections that are clearly out of sync.” I protest. I beg. I plead. I decide to get a second opinion from Andy, the drummer. I figure as a drummer and member of the band, he’ll be able to speak with authority on the timing and sync, and will clearly see it’s perfectly in sync. “Sorry mate, Sascha’s right – a good 80% of this is out of sync. The audio is off, the timing is off.”

I’m ready to give up. I have spent months and months of personal time on this project, all for the love of the project and tiny glimmering prospect of future profits from an on-demand sale. I protest with both Sascha and Andy further, but they are adamant. The video is out of sync and must be corrected before going out.

After a week absorbing this idea, I come around. Looking closely I realize they are correct. At least 20 of the camera angles need to be nudged forward or backward by a few frames to achieve sync. The problem is, having hard-burned the first two editing passes into ProRes 422 files, there’s no way to go back and make this adjustment. I will have to re-edit the entire 90 minute video.

THE EDIT: ROUND 4 (THE FINAL)

Having gone through hell and back with the embedded multicam sequences in Premiere, I know that is NOT the way to proceed. But at this point, I’m exceedingly familiar with the music. Then it hits me: edit blindfolded.

Instead of worrying about structuring the multicam sequences in layers, just setup one MASSIVE multicam sequence with every single angle, tapping the beat blindfolded. Literally blindfolded. After all the cuts are made in the sequence using the “add edit” command (I remapped this to the spacebar to make it easier to find while blindfolded), simply open the project, and then select a camera angle for each cut using the keypad (or the drop down selector).

The advantages of this method are:

You cut exactly on the beat and aren’t distracted by the visuals
You instinctively insert more cuts, thus creating a more engaging, dynamic video
You are not constrained to cut it live in real-time (we found that even with nine cameras, if you stopped the playhead after six minutes and then restarted the edit process, a noticeable one-second lag was introduced every time you made a cut, thus making it necessary to edit the entire live show start-to-finish in one pass)
Since there are no embedded multicam sequences, you avoid all the trouble there

The major disadvantage here is that playing back 38 cameras of multicam is nearly impossible for most computers. Even reducing the frame rate and using tiny 640×360 proxy files didn’t help.

But since I had already cut the set ten times and listened to the set countless more, I was so familiar with the shots, I simply took a screenshot of the multicam output, added numbers in photoshop, and placed this printout next to my monitor:

This fourth and final system worked. And it work GREAT. I could go back and fine-tune any of the edits very easily, I had a much faster, more engaging piece, and I was able to nudge entire tracks forward or backward to adjust sync at any time.

It was this final method that resulted in the actual product, which is available online at www.WeAreKMFDM.com. Or check out the portfolio entry on the Punch Drunk website.

Thanks for reading and good luck with your massive multicam project!

TAKEAWAYS

Run all your editing files on an SSD or SSD RAID. We like the Highpoint.
Put all your scratch disks and preview files on another, different SSD or SSD RAID.
Make sure your OS is on an SSD.
Use ClipWrap to make AVCHD files more manageable.
Avoid embedding multicam sequences into other multicam sequences.
Standardize all your multicam shots into one format, such as ProRes LT or proxy to do the actual edits. It’s very easy to replace proxy files with the full-resolution versions later.
If you do edit with proxy files, make sure they are the same frame rate. Many elements can be different – even size or aspect ratio – but frame rate should be the same.