consumer vs. professional formats, and 24p again.

[Heath McKnight]

ps-Link to the new HDV specifications:

http://www.hdv-info.org/HDVSpecifications.pdf

[Wayne Morellini]

I should have posted this over in that flash drive thread, but here it is. IBM is planning 100GB SD card (can go bigger, and make an array for higher data rates):

http://www.research.ibm.com/journal/.../vettiger.html
http://blogs.zdnet.com/BTL/index.php?p=1143&tag=nl.e539
http://www.tomshardware.com/hardnews...11_162423.html

As you can see (apart from the 3.2TB holographic disk plans from Toshiba, I think) lossless/higher data rate portable recording is soon to be viable within the coming year or so.

[Ash Greyson]

I never said current formats are not compressed, I said future PROFESSIONAL ACQUISITION formats will head towards less compression.



ash =o)

[Barry Green]

Yes and no.

Obviously professionals prefer less compression, that part would never be in doubt. But uncompressed? That's probably quite a ways away. Even HDCAM-SR uses MPEG-4 compression (at 440 megabits per second, so somewhere around 4:1 compression?)

AVC-HD isn't about being a "professional" format, but I do think it'll be adopted by professionals just like DV was. DV was never "intended" to be a professional format, but it undoubtedly was heavily adopted by professionals at all levels. I expect AVC-HD will garner similar acceptance (depending on the type of product the manufacturers produce!) And yes, compression is absolutely integral to any sort of functional workflow right now. With DVCPRO-HD I could have multiple streams of 1080 high-def, including transitions, running on my G5 in realtime, off the internal hard disk (no RAID necessary.) That simply could not happen with uncompressed at this stage of the game.

Less-compressed is of course better; that's why Sony developed MPEG-IMX (using I-frame-only) and Panasonic is going with AVC-Intra (50 megabits -- and perhaps more -- of I-frame-only) in their higher-end professional gear. And as capacities increase I expect that they will introduce milder and milder forms of compression.

But 50 megabits still represents a compression ratio of somewhere around 30 to one (and that's based on 8-bit, rather than 10-bit, and full-raster obviously).

Someday, yes, when capacity and bandwidth and processor capabilities all allow for it, uncompressed will be the way we trend. But it isn't going to happen anytime soon. To work with this footage in today's environment, and today's workflow, and today's IT capabilities, heavy compression will continue to be the order of the day for the foreseeable future.

[Ash Greyson]

It depends on how you define professional... I always say there will be some apps that will prefer convenience over quality. I mean if you are an ENG guy shooting a car accident, fire, football game, etc. I dont think it matters, those guys surely are pros in that they get paid to shoot but I am more talking about stuff that gets broadcast, CCed, run thru a post house, etc. I cant imagine shooting a highly compressed format for a music video, etc.

Barry you are talking about it being adopted because it can help overcome some of the technological limitations but what about when those limitations are overcome? That is my entire point, HDV, DVCproHD, etc. etc. etc. are really there not because they are the best, but because they are the best within the affordable technical specs. I just dont see a big move towards more compression, especially when the big networks are currently shunning such...



ash =o)

[Barry Green]

When such limitations can be overcome, then there's no need to perpetuate the compressed format.

That's what the EBU decided when they rejected 1080i as the HDTV format for Europe; interlace is basically a form of compression (a way to cram 50 motion samples in half the bandwidth) and since 720/50p works fine, they said "no more interlaced".

So yes, I would agree that professionals would prefer better image quality, and lossy compression doesn't help that any.

But the problem is, it's completely impractical to even start thinking about uncompressed HD. We don't even really do uncompressed SD; at 27 megabytes per second it would strain most hard disks to keep up with even one stream.

And by the time computers can handle the 150-megabyte or 200-megabyte per second data rate (and capacity issues) of uncompressed HD, we'll probably be on to the next, larger format. UHD. 4K. And those will require compression to make 'em workable.

We got there with audio -- 16-bit 48kHz uncompressed, and even 24-bit 96kHz uncompressed, are standard. But with video it's so preposterously more difficult. Which is why compression will probably always be with us, to some degree or other. The defining point is when it's good enough for widespread acceptance, and can be worked with.

I agree that less compression is of course better. I just don't see it happening anytime soon. Highly compressed is going to be with us for the foreseeable future. That's why AVC-HD is so interesting to some of us -- it appears that it may finally cross that threshhold of "good enough". The bandwidth is certainly low enough to be workable, and assuming that nVidia and ATI put hardware AVC decompressor chips on their graphics cards, the processor load may be low enough to make this a workable format. My only real reservation about AVC-HD would be regarding the long-GOP issues, but with twice the efficiency of MPEG-2 I'm expecting that will be less of a worry.

Highly compressed also smooths the inevitable transition to tapeless. I think the P2 cards work just fine for the limited capacities we can get currently (for some type of projects, but not for all.) But with AVC-HD you could store 5x as much footage in its best-quality mode... now that starts to be downright luxurious. 20 minutes in best quality on a 4gb card, 80 minutes in best quality on the 16gb card? And if you're willing to compromise yourself down to 12 megabits, now you're looking at 32 minutes on a 4gb card, and two hours on a 16gb card.

I think the mini-DVD application for AVC-HD is just silly, but I do applaud Sony for bringing out a 30gb hard disk model, pointing towards more capacity and more options. I just hope that when Panasonic brings out their SD-Card models, that they have the foresight to engineer multiple card slots. Having three or four card slots full of cheap off-the-shelf 4GB SD cards will mean that short recording times will never be an issue.

[Peter Ferling]

I think it's not so much the compression itself, but that compression technologies have improved (i.e. same visual quality, but smaller file size). There was a time that all compression resulted in a poor image. If you wanted uncompressed, it was very expensive, and required specialized gear, etc. The cheapest for SD, even five years ago, was a video toaster and NT workstation for about $15K base. Then you needed some expensive $35K or more SD camera's to actually realize the benefits of uncompressed.

You were lucky if your array of expensive barracuda drives lasted longer than six-months, and that workstation itself didn't go down every two or three weeks. The uncompressed extracted a huge penalty that frankly, wasn't worth it.

Today, compression is not a bad thing. Especially if you cant tell the difference in the end product as to how it was originally aquired. That is, say BetaSP vs. DV for corporate video. It made more sense for me to go the DV route and have a kit, camera, and PC that cost less then the price of a new $40K betacam, when the end result was the same, and the workflow more efficient.

The bottom line resulted in a wider profit margin as the overhead was substancially less. We're talking about a big difference, about $800 less per finished minute!

Granted, uncompressed may have it's benefits in post and green screen, but it didn't make sense, in fact seemed more like overkill, to have it for the entire project.

[Thomas Smet]

Compression doesn't have to be a bad thing. Look at Cineform. This codec is amazing and may come very close to uncompressed in terms of visual quality but at a super small datarate. In the future even if we could shoot and record uncompressed HD I think more people would rather use a codec like Cineform or better to have many times more storage space not to mention many real time streams for editing. Even if 200MB/S gets cheap remember that is for each stream. If you want to do a 4 camera multicam edit in FCP with uncompressed HD your hard drive will need to pump 800 MB/S sustained. That is nuts. I would rather have a codec that had no visual loss such as a wavelet based codec like Cineform at 10bits that would be just as good even for green screen work.

One question about an I frame only AVCHD codec. How much better would it be than DVCPRO HD at the same bitrates? Isn't most of the compression in the GOP length and changes between P and B frames? Wouldn't a AVCHD I frame only at 100 mbits/s be the same as DVCPRO HD at 100 mbits/s? Is there other areas of compression that would make it better even on I frame only? I kind of look at it like how mpeg2 I frame at 50 mbits/s is very similar to DVCPRO50. I never could figure out what the benefit was of mpeg2 I frame 50 mbits/s compared to just using a 50 mbit DV based codec.

[David Heath]

Originally Posted by Barry Green

That's what the EBU decided when they rejected 1080i as the HDTV format for Europe; .......and since 720/50p works fine, they said "no more interlaced".

I'll go along with nearly all you say in your last posts, Barry, but from a European perspective that above statement just is not true - 1080 is establishing itself as the dominant format over here, at least in broadcasting. The EBU did express a preference towards progressive over interlace, but certainly didn't reject 1080i - though their wish is eventually towards 1080p/50 for top end acquisition at least.

In the meantime, the two HD services we now have in the UK, from Sky satellite, and a trial service from the BBC, are both 1080 only. The BBC service is 1080i/25 for some types of programme (sport etc), but 1080p/25 for such as drama, carried psf.

[Barry Green]

Originally Posted by David Heath

I'll go along with nearly all you say in your last posts, Barry, but from a European perspective that above statement just is not true - 1080 is establishing itself as the dominant format over here, at least in broadcasting. The EBU did express a preference towards progressive over interlace, but certainly didn't reject 1080i - though their wish is eventually towards 1080p/50 for top end acquisition at least.

I base what I said on the EBU's technical review paper, offered here:
http://www.ebu.ch/en/technical/trev/...editorial.html

In that, they specifically said that they favor 720/50p (or 1080/50p) over an interlaced format such as 1080i/25. Now, they can't determine for the broadcasters what the broadcasters will choose, but they stated their preference rather strongly, even going so far as to quote Yves Faroudja saying "I am amazed that anybody would consider launching new services based on interlace. I have spent all of my life working on conversion from interlace to progressive. Now that I have sold my successful company, I can tell you the truth: interlace to progressive does not work!”

Now, they did recognize that "Some suppliers of HDTV equipment have complained that the EBU’s support for progressive scanning is damaging the case for 1080i/25 and for the 1920 x 1080 common image format." And they went on to basically say "hey, we think it's a bad idea to promote interlaced, but we recognize that some of you are going to do it anyway." They said "The EBU has no intention of starting a format war or prohibiting the use of 1080i/25", but they felt that such systems would be more appropriate for acquisition but not for transmission (aka "emission").

So going back to what I said: I said that the EBU rejected 1080i, instead favoring progressive (and, for today, 720/50p) for transmission. And that they said "no more interlaced." As the document I pointed to illustrates, they specifically are "firmly in favour of progressive scanning." They acknowledged that both systems will co-exist, especially for acquisition, but they specifically endorsed only progressive for transmission. They'd prefer 1080/50p, but until that arrives, they endorsed 720/50p.

Whether broadcasters are following that recommendation or not is, of course, another subject.

[Kevin Shaw]

Originally Posted by Barry Green

I don't doubt that these little AVC-HD cameras won't hold up as compared to an FX1. But an AVC-HD FX2 would be substantially better than an HDV FX1, of that I have no doubt.

I think we should reserve some doubt until we see how well the real-time AVC encoding chips in the cameras work, plus see how much recording media will cost and consider how we might work with footage from such cameras. If the footage is only slightly better and other aspects of the cameras are less convenient that might not be so great, especially if no current computer can effectively process the footage without converting it to some intermediate format. So maybe AVC will be a great thing which blows HDV away and maybe it won't; I wouldn't bet either way right now.

[Kevin Shaw]

Originally Posted by Ash Greyson

YOu proved my point... all thes "I will never go digital" guys are shooting RAW, not compressed... with more and more movies going thru extensive CC and post compositing, LESS compression will be the goal for ACQUISITION.

My point was that some people didn't think digital photography could ever rival film, and most have now realized otherwise. As far as compression is concerned, I think we'd all agree that less of it is generally better. But like I said before, if all video delivery is heavily compressed we don't need to shoot full uncompressed source to get acceptable results for typical delivery, and hence economics will play a significant role for the foreseeable future. Cameras like the SI-1920 point the way to affordable, modestly compressed video acquisition which should look "good enough" for most purposes without breaking the bank shooting uncompressed. Uncompressed editing may make sense, but uncompressed acquisition won't for anything but the best-funded projects.

[David Heath]

Originally Posted by Barry Green

I base what I said on the EBU's technical review paper, offered here:
http://www.ebu.ch/en/technical/trev/...editorial.html

I'm well aware of that document, and there is no doubt that the EBU (sensibly, IMO) favours progressive over interlace in principle. I just feel that saying that they had "rejected" 1080i is taking their position too far, and to quote from the document above:

".......... the EBU has recognised that 1080i/25 services will operate alongside 720p/50 services – and strongly hopes that 1080p/50 systems will eventually become the norm.

The EBU has no intention of starting a format war or prohibiting the use of 1080i/25."

I just didn't want non-European readers to get the impression from what you posted that Europe was becoming a 720p/50 only zone for broadcast, since nothing could be further from the truth. See http://www.bbc.co.uk/guidelines/deli...ats_v01_01.pdf for the BBCs own guidelines. Note the specific mention of 1080p/25 - they do not simply say 1080i.

Coming back on to topic, I agree with what Barry says in most respects about AVCHD, and it's likely good future, together with AVC-Intra at a higher level. The only exception would be regarding the mini-DVD based camera from Sony, which I feel is far from silly, at least in a consumer camera. I suspect a lot of such cameras are used for recording family events, and most of todays tapes are never edited, but rather put on a shelf for looking at sometime in the future. I doubt Joe Public is likely to be able to change his behaviour, and hence shoot to disc and play back on a future set top player will hold for them big appeal.

Other users may well prefer multiple SD card slots for very good reasons (for them). My own preference would be for a hybrid SD/miniDVD camera, and the ability to use either or both medium as appropriate.

[Wayne Morellini]

You may be aware that I am, oddly, quiet here, there is just not much to say. I actually prefer to talk about the AVCHD cameras/technologies ;).

In the Digital cinema camera projects we realised a number of things (and if you read my thread you will see that people actually do have uncompressed acquisition running, at a price).

Hard drives can sustain 50GB per second, with sizes like 500+ MB per drive. That is three drives for 1080i/p 8bit 4:4:4, or around one drive for 720/24p uncompressed.

If you look at lossless compression most run out of steam after an average of 2:1, so halve the drives with a good codec. At the moment I am looking into a codec by digital Anarchy (to see if it is credible and capable) that promises lossless at 4:1+ average.

In the Digital Cinema camera projects we quickly realised that Bayer filter actually would give use 1080p24 8bit at under 50MB/s. Bayer has the ability to render a good 4:4:4 approximation, and is used by the major Cinema camera manufacturers (though not Sony) where they can turn on the flood lights ;). From discussions on the group, Cineform developed a bayer version of their codec, that gets, maybe, upto 6:1 on the latest edition. At the moment they are trickling out something like 9 MBytes per second for 1080p footage. This is supposed to be visually lossless, but the way bayer requires preciseness for interpolation, I would not be surprised if it is close to lossless.

I have long ago known that you can fit cineform like codec on pocket cameras, it would be far better to scrap h264 and go with cineform for certain levels of the video camera industry (and render out to a distribution codec). I have had discussions with them, and if anybody would like to do it, and back it up with money of course, it is possible to do it. I would imagine that a fair few of these future h264 cameras could be reprogrammed to do this. On the extreme edge of visually lossless, or adjusted for lossy, cineform could, perhaps, keep up with H264 for a lot less processing cost.

With my own compression codec plans, I estimate I could do a true lossless 720/25p 8bit in about 20mb/s. Cineform could be adjusted for this, or some other lossless codec. This is an extreme in one of the techniques, but I don't think the extremes of what is possible in lossless compression. You can all wipe up the coffee you spat out on the floor now, I am reasonably confident that this can be done, but who by, I don't really wish to.

[Douglas Spotted Eagle]

Originally Posted by Kevin Shaw

So maybe AVC will be a great thing which blows HDV away and maybe it won't; I wouldn't bet either way right now.

Smart man.

It's interesting that some folks are willing to bet on the "potential" of the format vs what will be delivered by manufacturers. We all know that MPEG in camera isn't using nearly the total potential. There was a camcorder heavily marketed over the last year that was *marketed* on it's potential and disappointing in actuality. What a format or spec can do and what is delivered in the physical box are usually two very different products.