« Back to the Product Team Blog

Automating Campaign Videos with Twilio and FFmpeg

Last year, we rolled out a new way for schools to raise money: Hudl campaigns. Online donation pages are nothing new to the web, so we went to the drawing board to find a way to make ours stand out. What we came up with was a compelling video with highlights from the team, great transitions and effects, and a heartfelt interview from the head coach. To test out our new idea, we manually crafted our first video used in Poteet’s Campaign by hand-picking which highlights to use in the video, calling the coach ourselves to record an interview and applying the effects in-house.

The campaign was a great success and coaches and fans were loving the new page. We wanted to see if we could fully automate the video production. We knew that scaling this out for the tens of thousands of teams on Hudl was critical. Twilio and FFmpeg made that easy.

Recording the coach interview

We wanted the interview process to be as easy as possible for our coaches. It needed to fit around their busy schedules and it needed to be completely automated. After digging around online, Twilio seemed like our best option. Their developer friendly API, .NET library and highly customizable markup language, TwiML, made the implementation a breeze. The workflow went as follows:

  1. A coach enters his phone number online and clicks “Call Me.”
  2. Our server sends a request to Twilio to initiate the phone call.
  3. Twilio then requests the TwiML for the call which defines which automated message to play and redirects to another page to begin the recording.
  4. After hearing the instructions and the question, this TwiML starts the recording after, specifies where to redirect upon completing the recording, and where to redirect if no response is heard.
  5. The coach enters “#” to save the recording and Twilio makes a request to our server with the URL to the recorded file which we then save.
  6. During this entire process, each Twilio request updates the status of the interview for the coach and at the end lets the coach preview the response. Interview Screen

Creating the polished video

At Hudl, we’ve been using FFmpeg for years to encode game film, so our first move was to start researching all the different things FFmpeg could do for us. We were more than satisfied with its broad range of functionality. We then searched around for any .NET libraries for interacting with FFmpeg. After not finding anything, we then set out to build our own framework called: Hudl.FFmpeg. The framework contains wrapper classes for each parameter that FFmpeg supports. To generate an output, you begin by adding input objects (either video or audio) to an output command and then applying any number of filters or output settings on those objects. The framework will then translate each of the output commands into the required syntax which is fed to the FFmpeg executable. Keep an eye on this blog. We’re going to open-source Hudl.FFmpeg, but for now I will share with you the C# code used to render our campaign videos.

  1. Start by concatenating all the highlights together and applying a custom cross-fade transition between highlights.
  2. Overlay a vignette image and blend a film grain video effect into the video.
  3. Tweak the color balance of the video to make it stand out and then end the video with a 2 second fade out transition.
  4. Concatenate all the interview audio files and convert them to M4A.
  5. Mix the coach interview audio with an inspiring audio track playing in the background with custom volume levels.
  6. Combine the rendered video with the rendered audio and output a 480p final version.
  7. Render a separate 240p version for mobile devices.

Conclusion

Almost a year later, we are still hearing from coaches, athletes, and the community about how touching and inspiring these campaigns can be. Lee’s Summit, MO had a recent campaign that raised over $10,000 in just 30 days.

Without our automated process for generating these videos, we simply wouldn’t have had the resources available to roll this out nationwide. It truly has been an awe inspiring process to go from something that took one person a few hours to manually craft into something that can be rendered automatically in around 10 minutes. To put that in perspective, we’ve had over 1,800 videos created saving over 3,600 person-hours of time by automating this process.

comments powered by Disqus

Engineer the Smoothest Video Tools Ever Created » Apply Now