A while back, I wrote a little article on screen resolution which hopefully enlightened readers on what it actually means, and demystified the world behind the jargon, which is so frequently exploited by TV salesmen to sell you expensive TVs. Part of the inspiration for that article was this XKCD comic, and recently there has been a lot of talk in the movie industry about frame rates, which leads me to explaining the mouseover text in that comic.
The complaint from tech-heads is often two-pronged, first they complain about resolution. This complaint is usually leveled at TV, in particular the fuss that is made about HDTV, which is not actually particularly high (see above-referenced article). The second concerns frame rates. There are many standards in existence these days, but only two relevant ones – 24p, and 30p.
Very old TVs (and some new ones) use what is called interlaced scan, which is where the odd and even horizontal lines of the screen come on and off in an alternating pattern. Anything new (read: digital) would be in progressive scan, where the entire image changes with the frame rate. The “p” in the above standards refers to progressive scan. 24p simply means progressive scan at 24 frames per second.
Standard HDTV is usually broadcast in 30p. Generally 1080/30p, which means 1080 lines of vertical resolution at a progressive scan of 30 frames per second. The complaint is that the industry standard for feature films is 24 frames per second. It has been this way for a very long time. I’m not sure why they chose 24 frames per second, but I have a feeling it had something to do with the (primitive) technology of the time. It is certainly possible for our eyes to distinguish between 30 and 24 frames per second and it is often suggested that, since TV is 30 frames per second, it has “trained” our eyes to associate the higher frame rate with TV, and thus movies are condemned to staying at 24p otherwise our brains would automatically think that they are “TV”. James Cameron recently made ripples in the film making industry by suggesting that he might try shooting at 60fps.
After thinking about this for a long time, I think I have figured out the problem and it doesn’t have anything to do with the frame rate per se. The key to understanding what’s going on here is understanding the oft-confused difference between frame rate and shutter speed. We begin (as I did in the article on resolution) by examining the most fundamental piece of hardware in our equation – the human eye. In the previous article, I established a reasonable theoretical maximum resolution for certain sizes of screens, based on the maximum acuity of the eye and the anticipated viewing distances between the eye and the screens. This, however, is slightly more complicated.
The human eye perceives things in a certain way. The maximum aperture of our eyes is roughly equivalent to f/2 on a camera lens, and the resolution in the center is significantly higher than towards the edges. This is part of the reason that pictures like the one above, taken with a wide-aperture lens on a professional SLR camera are generally more appealing than photos taken on cheap point-and-shoot compact cameras where everything seems to be in focus – the former is much closer to what our eyes actually see. Selective focus using what’s called “shallow depth of field” is more than just a simple trick of a photographer to draw your attention to a certain part of the picture – it simulates what our eyes actually do when we draw our attention to a certain portion of our field of view.
Frame rates are a trickier thing to figure out than the relative aperture size of the human eye however. The world (if physics is to be believed) happens as a continuous movement – one could say at an “infinite” frame rate. In high school, we once conducted an experiment where we set up a camera shutter so we could look through it and try to recognize an image. We found that most of us could recognize images when the shutter speed was as high as 1/250 of a second, but beyond that it was too short a time. Nobody could recognize anything past 1/500 although there are likely a handful of people in the world who can. However, many experiments have been done where subjects were shown filmed sequences at over 100 frames per second, and they described it as “artificial” as well as sometimes making them feel ill.
I think the real problem here, is that when motion film is shot at say 100fps, this necessarily means that each of the individual frames is taken at 1/100 of a second or faster. This takes away a lot of blur. In the same way that our eyes naturally blur out most of an image if we aren’t looking directly at it, we also blur out a lot of movement when we aren’t directly tracking it. Traditionally, 24fps film is shot at what is called 180 degree shutter. (see diagram below)
This means that the shutter speed is usually 48fps. Motion picture lenses are usually very wide aperture (and very expensive) to get that selective focus “look” and the exposure is controlled using a combination of extensive lighting setups and neutral density filters (sunglasses for the lenses). Believe it or not, almost all film is shot at this shutter speed. There are some notable exceptions – in Chariots of Fire, for the scene where they’re running along the beach, they painstakingly cut out every second frame and duplicated the frame that was not cut, to give the effect of 1/48 shutter at 12 frames per second. In the more recent film Gladiator, for the fight scenes, the shutter speed was approximately doubled, but the frame rate was kept the same (1/100 shutter at 24 frames per second). The interesting thing to note here, is that both effects are noticeable, even if they weren’t immediately apparent.
The point here is that, for relative motion, we perceive things with a blur. Changing that blur in any way will produce a noticeable effect. HDTV in 30 frames per second was probably shot with a shutter speed of 1/60 of a second, which is significantly faster than 1/48. In addition, TV cameras have relatively small image sensors, usually 2/3″ format, which has about 1/17 the area of a full-frame sensor, which translates to having much less ability to offer selective focus in the way a proper motion picture camera can (part of the reason for this, is that with live TV, it makes it easier to keep everything in focus, whereas in motion picture shooting, everything is meticulously planned, including the exact point of focus, and it usually takes 3 people to operate 1 camera, whereas in TV each camera is expected to only require one person to operate).
So really, the problem has nothing to do with the frame rate itself. Our eyes are clearly OK with perceiving an infinitely-high frame rate (a.k.a. the real world) but the shutter speed at which the film is recorded must, in most cases, simulate our ordinary perception of the world, including blur, otherwise our minds will reject it as “fake”. The practical problem with this, is how to make it possible to shoot at frame rates around 1/40 and 1/50 of a second, yet project at frame rates where each frame may only be visible for 1/60 of a second or less. One promising solution that has been tried is software interpolation (basically “guessing” the intermediate frames) which seems like a pretty good idea. However, the software is not sufficiently advanced to guess very well – typically hard edges and contrasts are what gets interpolated (because it’s easy for software to “see” these things), but the key to achieving the proper effect is to also interpolate the blur.
As long as motion is in any way mechanically-captured, it will be impossible to project film at a higher frame rate than the reciprocal of the shutter speed (e.g. 60fps for 1/60 of a second). However, there is promise in the area of electronic motion picture capture, although the bandwidth capacity for image sensors still has to increase significantly for this to be possible (as well as processing power at the point of capture, not to mention the bandwidth increases necessary for higher resolutions, like I mentioned in my other article). Still, the rate at which technology is advancing in this area is breathtaking to say the least, and I feel confident that it is entirely conceivable within our lifetimes that we will see maximal resolution films in very high frame rates that still look “natural” with proper out-of-focus blur AND motion blur.