I’ve received a lot of great feedback on the travel-lapse that I posted. There were enough people wanting to know more details, that it seemed worth writing a second post about.
For reference, here is the original video. The shot I’m writing about is just the second to last shot of the sequence:
The thing that is interesting about this time-lapse, or travel-lapse as I have been calling it, is the sort of motion that it displays. There are lots of moving time-lapses shot from vehicles, trains, cars, etc. but they’re closer to stop-motion than real time-lapse. They show accelerated movement through space but are not stable enough to show smooth motion of objects in the frame, such as clouds for example. You can walk down the street taking a still photo every second and then string them together into a video clip for a very cool effect – it’s a variation on stop-motion. In this sort of shot you would see the buildings along the street in every shot but the buildings aren’t tracked and stabilized in a way that we see smooth, continuous motion such as the movement of shadows across the face of them or the smooth growth of clouds in the background. It’s a seemingly subtle distinction but a dramatically different effect.
I can’t say for certain that I’m the first one to do this. If I can do it on my computer in my home studio then others could also. But I monitor the industry newsletters and am pretty on top of trends in time-lapse techniques and I’ve never seen this done before. The point of course, is not that this is the first one, but that this is possible. Every day it seems the horizon of what’s possible for individual artists and filmmakers gets pushed further out. This particular technique is not, in theory, that difficult. This example was significantly more challenging than other similar shots you could set up because it was shot from a violently moving boat without a tripod and with a wide angle lens. The motion presents challenges for stabilization and the wide angle lens increases those challenges because the wide angle distorts the image and, because of the large movements of the camera, it distorts it slightly differently for every shot. This means that not only do you have to stabilize objects within the frame but you have to un-distort them as well. So if this technique can work on a shot like this, it can be easily achieved in other circumstances.
I tried all sorts of corner pinning to achieve this but was unable to get an acceptable result. The thing that made this shot possible in the end was After Effects’ Warp Stabilizer. This tool falls squarely into the growing list of tools and effects that work auto-magically. There really is no other word for the power built into these effects. Things like the roto-brush and content-aware fill are completely redefining how we work and what can be accomplished. It’s shocking how fast the face of digital video creation is changing. It’s both thrilling and intimidating at the same time. A discussion of this really deserves its own post at some point so for now, I’ll try to stay on track with time-lapses and stabilization. I want to mention a couple of things first though about how I’m working now and why.
About a year before Final Cut went belly up, I jumped ship to Adobe Premiere Pro. My principal motivation was better integration with the other software in the Adobe Creative Suite. I often do all of the widely varying tasks that comprise video production and post-production by myself. This involves walking a fine line trying to balance your return on time invested in various technologies. We all want to take our projects to the next level and with the technology we now have at our fingertips that’s possible – but not always practical. You can’t do it all – and certainly not in any reasonable amount of time. As my editing skills and knowledge grew, I began to rely increasingly on programs like After Effects and Photoshop to achieve things that aren’t possible in an NLE alone. Things like color-grading and stabilization and re-timing are much better done in After Effects than any current NLE. And stabilizing, for example, is something you’re going to have to do some of if you want to maximize a small budget – much easier to learn a little After Effects than rent and operate a gyroscopic stabilizer. Going back and forth between Final Cut and After Effects is tedious and inefficient and that’s only gotten worse with Final Cut X. If you’re making films in a larger studio environment, it’s easier and often even necessary to export out sequences and xml’s for the people that are handling the compositing or coloring or animation or what-have-you, but as a one-man-band it’s a big advantage to be able to work entirely within one suite of tools. And I don’t think there is a better suite of tools available than Adobe Creative Suite. To my surprise, I found Premiere Pro not only plays nicer with other software but is actually a better NLE in my opinion. And one of the biggest of many other advantages I’ve found is that the keyboard shortcuts for my most used software now all have a common vocabulary. Whenever possible, tools and functions across the Adobe Suite share commands. There is a much-touted preset in Premiere that lays out the keyboard commands just as they are in Final Cut. This is handy for switchers but my recommendation is to just jump in and learn the Adobe shortcuts. They’re easy, especially if you know anything about Photoshop or After Effects, both of which I consider necessary for any video post-production. Hanging on to the Final Cut keyboard commands will just slow you down in the end.
All of this is relevant to the time-lapse discussion because imaging technologies are merging. Still photo and video are overlapping more and more all the time. RED and modern DSLRs are accelerating the blurring of the line. Time-lapse video (all video for that matter) is made up of still photos and Photoshop is the undisputed standard for still photo editing. The advantages of Creative Suite are evident across the spectrum of post-production but time-lapse workflows are a particularly good example. Like many things, time-lapses can be created in a variety of ways and using a variety of software. People are making amazing stuff with the humblest of tools and the product is the final judge of any given method’s merit – so I don’t mean to be evangelical about Creative Suite. There are many good tools out there and the landscape is changing all the time. But given a defined set of objectives, there will be more effective and less effective ways of going about a given task. I don’t know of any tools currently available for creating time-lapses that can equal the combination of Photoshop and After Effects.
This post is not meant to be a tutorial so what follows are very pared down steps. Each of these steps requires a whole set of skills that I won’t elaborate on here – these are just rough outlines of the process that assume you’re familiar with a wide variety of editing tasks.
My basic workflow for time-lapses is as follows:
1. Open all Raw images in Camera Raw (There is no good reason to ever shoot anything other than Raw for time-lapses) which is a little companion app that comes with Photoshop. You can use Lightroom as well. Lightroom is an excellent program and I use it for all of my non-time-lapse photography. The choice depends mostly on how many time-lapses you shoot. Time-lapses require a lot of large images and they take up a ton of space and can start to overwhelm a database like Lightroom if you have a lot of them. If you do use Lightroom for time-lapses, I strongly suggest creating a second database just for time-lapses and maybe even for individual projects. The other thing is that once I have “developed” the raw images, I export these as tiffs and keep both the complete set of raw files aw well as the tiffs. If you have a very fast machine, you might not need to export to intermediate tiffs and can just create the time-lapse directly from the raw files. I have a fast 8 core machine with 30 GB of RAM and a souped-up graphics card and I still find it more efficient to make intermediate tiffs. But After Effects can support super-hi-res raw files and in the future, that will be the workflow. For right now, this makes using Lightroom a little more complicated. I choose to store all time-lapses in folders labeled by date. In either case you need a program that can batch edit Raw images.
2. Make edits to Raw images and crop to desired resolution. 1920 x 1080 is fine unless you want to do any digital panning, zooming, etc. on the shot, in which case you should export at a higher res. Export all images as 16 bit, Pro Photo, tiffs.
3. Import image sequence into After Effects. In some cases After Effects is simply used to export the image sequence as a video file. But even in this most basic scenario it is significantly better than just using Quicktime. Just for starters, you won’t have to deal with the Quicktime gamma shift problem (google this if you haven’t encountered it), you’ll have access to much more powerful exporting options and After Effects is color managed!! This last point is huge! You want to color correct your time-lapses before you compress them and you definitely want to do this in a color-managed application. In addition to these points however, very often you will want to use some of the tools inside of After Effects to enhance the shot. This might include stabilization, additional color grading or masking, digital motion control, re-timing, etc.
The basic steps that I ended up using to stabilize the moving travel-lapse were:
1. Export all images as full-resolution tiffs without any cropping. This is important for the process to work. In fact the process relies on being able to do various stages of stabilizing and cropping and still end up with a 1920 x 1080 image.
2. Import the tiff sequence into Mocha and track the entire shot. Because of the large movements from frame to frame, much larger than you could ever get in actual video, you have to manually increase the size of the search area for the tracker. When the tracking is finished, set Mocha’s “surface” to cover the entire frame and then export both the corner pin data and the transform data to text files.
3. Open the image sequence in After Effects and use the Mocha data to do an initial stabilization. You can do this manually by copying and pasting the data into the comp if you’re familiar with basic stabilization techniques or you can use a handy little program called Mocha Import by Mamo World. It’s available for a donation of whatever you think it’s worth here.
4. The stabilization will leave black margins around your comp. Resize the comp to 2880 x 1620 and adjust the scaling of the image sequence to fit inside the comp without showing any of the black margins. This is important because any black borders will mess up the Warp Stabilizer in the subsequent steps. Also, don’t scale above 100 percent! Export the comp in the format of your choice. I use ProRes 422.
5. Import the video that you just exported back into the project and run the Warp Stabilizer on it. I use all the highest quality, most processer intensive settings. Make sure that the output is set to Stabilize only. Adjust the comp size to 2400 x 1350 and scale the video to again fit in the comp frame without black borders. Again, don’t scale above 100 percent. Export.
6. Import the new video and run the Warp Stabilizer one more time in the same way as before. Resize comp to 1920 x 1080 and scale video to fit. If, at this point, you are running out of resolution, you can use one of the various techniques to fill any black borders without resizing the video such as using a motion tile filter, placing a duplicate copy of the video scaled slightly larger behind the first or using Warp Stabilizer’s synthesize edges feature.
In theory, you shouldn’t need these multiple passes of stabilization but I found that for this shot, each successive pass smoothed out the motion a little better and parts that the stabilizer couldn’t fix the first time were corrected on the second or third time. The ability to do this sort of stabilization opens up all sorts of possibilities. In the very near future (it’s already begun) I think digital image stabilization will start to take the place of expensive and bulky image stabilizers for cameras. As long as cameras are shooting high enough resolution, many shots can be achieved using stabilization in post. Personally, I don’t see much need for 4k production as far as delivery is concerned. But 4k (and up) imaging does make a lot of exciting things possible for regular 1080p output. All of this is taking us toward greater reliance on post production tools. For better or worse, the days of getting the shot completely in camera are coming to an end. In this new workflow, being able to move your work easily between various programs and formats and incorporate stills and video and stop-motion and time-lapse and animation and everything else is a big advantage. I’m currently nearing the end of post-production on a film about the Bahamas that I’ve created entirely by myself. The film has a polish that I could never have imagined a no-budget, solo project ever having and that I could never have accomplished in Final Cut. The point of this post was not to cheerlead for Adobe, it just so happens that for time-lapses, and for other video in many circumstances, Creative Suite gives me a whole new level of possibility and efficiency. Another hugely important factor that I haven’t mentioned yet is that since CS5, Premiere Pro can do real-time, accurate YUV to RGB conversions. It’s hard to overestimate the significance of this and I’m surprised that it hasn’t been more widely discussed. For an excellent summary of why this is important, check out this link at the Pro Video Coalition site. This is the best explanation I’ve come across of critical video/color monitoring and is essential reading for anyone involved in any kind of post-production.
For anyone interested in learning more about time-lapse techniques, I’ll be giving a lecture that’s free to the public at the Rocky Mountain School of Photography in Missoula on March 22nd. More info is available on their website here. I’ll also be teaching a couple of classes as part of their Summer Intensive Career Training program this summer. Feel free to contact me or comment with any questions or thoughts.