Basics on video codecs and formats

[Luka De Smet]

Long time ago, in collage, I did learn some basic video editing. I'm trying to get back up to date, but several years are passed, the internet is now a important aspect aswel as high definition. I do manage, but need some clear and correct information about codecs en formats.

It seems impossible to find non conflicting information on the internet, and I also can't find books nor refresher courses on the subject. So I do hope someone here link me to correct information and books?
And explain me the basics.

I do know there are several codecs and its not easy to summarize.
There are delivery and editing formats.
Internet and offline formats
steaming and real-time playback (vj'ing) formats
ect

But I don't know whats what
Anyone care to help me out?

[Seth Bloombaum]

Somebody ought to write a book - or a web site! Sorry I don't know of one, but, the wikipedia page on codecs is not bad for what it covers; mostly intermediate and delivery codecs. At the bottom of that page you'll find links to several other relevant pages.

Briefly, though, you need to add to your list:

Acquisition codecs

Editing codecs

Digital Intermediate codecs

Delivery codecs

It used to be so simple in DV, when we'd use the acquisition codec for everything but delivery. Today, some workflows might use 4 codecs, more common is two or three.

MPEG-4 / h.264 is currently the best widely available highly compressed codec. At different bitrates, it's in use in cameras (AVC), can be (painfully) edited, and used for delivery. And we see more and more hardware support for compression and decompression...

[Giroud Francois]

look at video as plain data. Lots of data.
so to store or transport it, you need to compress.
The bad thing with video is it is sequential data. It means that you need to receive in the same order that it has been sent, so timing has its importance.

That is why in modern codecs , you got a clock concept embedded. If you loose some data, you do not really need to know how much you loose, but how long.

Basically some codecs still respect the old concept of pictures and frame and treat each picture or frame separately They are called Intraframe codec. These are codecs for recording or storage, giving nice result but require huge space.
DV, MJPEG, JPEG2000, some type of MPEG2 or MPEG4 are in this league.

The other part of codecs are streaming codecs. They consider video as stream of data.
This include most codec used for transport from the satellite to your TV for example.
They are made to be very resistant to data loss or corruption and highly compressing data.
HDV (a form of MPEG2-TS, TS standing for Transport Stream), Most of MPEG4 codec.
So they usually compress pictures by packets (or group called GOP, for Group Of Picture)
in a packet, you got usually one reference picture that is fully encoded, the other pictures being only expressed by their difference with the reference picture.

The fact is that most current MPEGx codecs are highly configurable so you can turn a transport codec that was designed to send video across the air into a recording codec embedded into a camera.
You just need to tweak some parameters to totally change it.

the main parameters to consider are
Bandwith: the amount of data per second that is delivered.
Picture compression: if it is Intraframe or GOP based, and if GOP based you need to know the GOP structure.(length and type of frames)
a GOP usually starts with a complete picture (the I frame), then is followed by by some other type of frame (B, P), but Wikipedia has a nice page about that.
Group of pictures - Wikipedia, the free encyclopedia
Number of bit used to represent data (usually 8 or 10)
Color reduction : A full signal is defined as 4:4:4 for the three parameter Y:U:V (Y luminance, U V color coding)
Most of codec are going to code 4:2:0 meaning that you got a black/white image at full resolution ,while colors are coded on 1/4 resolution. The eye is weak enough on dealing with color to be fooled by such trick. Professional codec are usually going to 4:2:2.

10 years ago, we got intraframe only codec in 4:2:2 and we were struggling with huge data file and bandwith difficult to handle by our slow processor. Capture has to be done in real time. At this time it was not data transfer , but more acquisition of an analog signal into a digital form.
Then we got DV that makes us jump into digital video, but we also downgraded from 4:2:2 to 4:2:0 (for european) or 4:1:1. Files were still huge, but since it was digital, there was no loss in the transfer, and expensive capture card was reduced to a simple high speed serial port (firewire).
In the same time, VHS was replaced by DVD and Mpeg2 start to rule.
Unfortunately, our processor were far too slow to encode Mpeg2 in realtime, so converting DV to mpeg2 was a very long process.
Today, Mpeg2 still rules for DVD and even Blu-ray, DV is slowly disappearing. Not because it was bad, but simply because it was designed to be stored on tape.
Most camera you can find now are storing video on some form of digital memory like memory cards or hard disk. There is no more capture, but file transfer at very high speed.
today you can transfer one hour of video in 10 minutes.
At the same time we got digital storage, we see Mpeg4 replacing Mpeg2 , DV and HDV (a form of Mpeg2 that was used to extend the life of tape on camera, bringing high resolution.)
Today, most of proprietary codecs developped by big names (Microsoft , Apple, Adobe, Canon, JVC, SONY) are using some type of Mpeg4.

[Luka De Smet]

Thanks for the information, you helped me on my way.
But I somehow expected more, the lack of reactions demonstrates how difficult this subject is.

[Paul de Vries]

Try this: Xiph.org: Video
"This first video from Xiph.Org presents the technical foundations of modern digital media via a half-hour firehose of information"

[Luka De Smet]

Thanks Paul, will check it out as soon as I got some time!