AVIs > 2 GB? at DVinfo.net

Peter Moore · December 22nd, 2003, 08:36 PM

I have a small problem, it's very technical but I hope someone here might know.

I'm generating Matrix-code AVI files using a C++ program I wrote. I'm using the Video for Windows functions. Everything's fine, except when the video is > 2GB (which iit is because they have to be uncompressed, HD - long story).

Anyway, it seems to me the problem is that somewhere there is a "size" variable that is a signed long (31-bits), the maximum value of which can be 2^31.

Short of manually creating uncompressed AVIs, which I can do, does anyone have a better idea?

Thanks a lot!

Adrian Douglas · December 23rd, 2003, 07:03 AM

Peter, are you using Win 98 or the FAT32 file system. Win98 had a problem with files over 2GB and sometimes so does FAT32. If your using W2k or XP try changing your file system to NTFS and give that a go.

Peter Moore · December 23rd, 2003, 09:25 AM

Hey Adrian,
Nope, it's NTFS. I long ago learned about the file size limitations of FAT32, so all my drives are NTFS. This is a programming issue. The files physically get larger than 2 GB, but nothing will play them properly.

I got the AVI file spec from MSDN. Making uncompressed AVIs (which I'll then compress with Vegas) shouldn't be too hard.

Rob Lohman · December 23rd, 2003, 09:45 AM

Peter,

The problem is probably in the matrox codec that is being used
to playback your file. I think one codec of matrix supports spanning
over multiple files, but I'm not sure.

I'd output uncompressed or just plain AVI DV. Keep in mind that
to have a > 2 or > 4 GB AVI file you will need to write a TYPE 2
AVI file!!! **NOT** a type 1. I don't know what the technical
difference is, but I do know you need to use the type 2 format.

Peter Moore · December 23rd, 2003, 10:21 AM

Ah, then that's almost certainly the problem. The VFW SDK probably is only creating Type 1 files, not Type 2, but I believe the AVI specs I got from Microsoft explain how to do type 2. Or I can probably find it one way or another.

Outputting multiple files is a great idea, which I can most certainly program my Matrix raining code gemerator to do. Thanks for the suggestion!

Colin Browell · December 23rd, 2003, 10:24 AM

It does indeed sound like the signed int problem that Peter is describing. AVIs are RIFF formatted with (originally) a single RIFF section occupying the whole file. The RIFF header contains a 32 bit length field which will only allow files up to approx 2 GB.

Anyway the format was extended (Open-DML standard) to allow multiple RIFF sections and therefore allow a MUCH bigger file size.

The VFW AVI functions do not use the Open-DML extension and so are stuck with a maximum file size of 2GB.

I am using DirectShow which does not have this limitation.

If you need to stick with VFW then you may be able to find some OpenDML AVI parsing code somewhere on the net - in VirtualDub for example.

Rob, I'm not sure what you mean but I *think* that the type 2 AVI file that you are referring to is the Open-DML standard (rather the the DV type 1 or 2)? If so it should really be referred to as an OpenDML AVI to avoid confusion with the DV types which are unrelated.

Peter Moore · December 23rd, 2003, 12:26 PM

Hm, interesting. I am actually not using DV, but uncompressed hi-def (otherwise this wouldn't be an issue). That's good to know DirectShow doesn't have this problem. Maybe I'll just use that, should be easy enough. Thanks a lot.

Colin Browell · December 23rd, 2003, 01:05 PM

Quote:

Maybe I'll just use [DirectShow], should be easy enough.

I found quite a steep learning curve with DirectShow, particularly if you are not doing something straightforward. It can be pretty frustrating to work with. Good luck.

Rob Lohman · December 23rd, 2003, 01:36 PM

I was talking about the AVI's you can create with DirectShow
which Microsoft itself calls TYPE 2 AVI's. Sorry, I didn't make up
the name <g>. I think you are correct in saying that these are
using OpenDML extension indeed (but that is a bit hard to
verify?).

VFW is always TYPE 1 and can indeed not playback the larger
files (in theory it should not even create larger files). DV type 1
and type 2 is a different thing (audio in DV stream or not).

Confusing world...

Peter Moore · December 23rd, 2003, 03:30 PM

Colin,
All IU'm doing is generating bitmaps in memory (actually DirectDraw surfaces, because they're fast, but all the same) and turning them into movie frames. It's extremely easy to do that with VFW - seems straightforward enough. Any suggestions for approaching that problem?

Rob,
Yes I read the OpenDML spec and that's exactly what AVI "Type 2" is as MS calls it. Basically all they do is have multiple RIFF chunks in the same file, each of <= 1 GB, so that a legacy player can read the 1st GB worth and advanced players can read the whole thing.

Colin Browell · December 23rd, 2003, 04:41 PM

Ah yes, there's quite a good explanation at

http://groups.google.com/groups?hl=e....freeserve.net

Sounds like your app may be quite simple then. You may just have to write a source filter and feed it into a file writer.

December 22nd, 2003, 08:36 PM	#1
Peter Moore Trustee Join Date: Oct 2002 Location: Arlington VA Posts: 1,034	AVIs > 2 GB? I have a small problem, it's very technical but I hope someone here might know. I'm generating Matrix-code AVI files using a C++ program I wrote. I'm using the Video for Windows functions. Everything's fine, except when the video is > 2GB (which iit is because they have to be uncompressed, HD - long story). Anyway, it seems to me the problem is that somewhere there is a "size" variable that is a signed long (31-bits), the maximum value of which can be 2^31. Short of manually creating uncompressed AVIs, which I can do, does anyone have a better idea? Thanks a lot!

December 23rd, 2003, 07:03 AM	#2
Adrian Douglas Trustee Join Date: Oct 2001 Location: Chigasaki, Japan. Posts: 1,660	Peter, are you using Win 98 or the FAT32 file system. Win98 had a problem with files over 2GB and sometimes so does FAT32. If your using W2k or XP try changing your file system to NTFS and give that a go. __________________ Adrian DVInfo.net Search for quick answers Where to buy? From the best in the business...DVInfo.net sponsors

December 23rd, 2003, 09:45 AM	#4
Rob Lohman RED Code Chef Join Date: Oct 2001 Location: Holland Posts: 12,514	Peter, The problem is probably in the matrox codec that is being used to playback your file. I think one codec of matrix supports spanning over multiple files, but I'm not sure. I'd output uncompressed or just plain AVI DV. Keep in mind that to have a > 2 or > 4 GB AVI file you will need to write a TYPE 2 AVI file!!! NOT a type 1. I don't know what the technical difference is, but I do know you need to use the type 2 format. __________________ Rob Lohman, visuar@iname.com DV Info Wrangler & RED Code Chef Join the DV Challenge \| Lady X Search DVinfo.net for quick answers \| Buy from the best: DVinfo.net sponsors

December 23rd, 2003, 01:36 PM	#9
Rob Lohman RED Code Chef Join Date: Oct 2001 Location: Holland Posts: 12,514	I was talking about the AVI's you can create with DirectShow which Microsoft itself calls TYPE 2 AVI's. Sorry, I didn't make up the name <g>. I think you are correct in saying that these are using OpenDML extension indeed (but that is a bit hard to verify?). VFW is always TYPE 1 and can indeed not playback the larger files (in theory it should not even create larger files). DV type 1 and type 2 is a different thing (audio in DV stream or not). Confusing world... __________________ Rob Lohman, visuar@iname.com DV Info Wrangler & RED Code Chef Join the DV Challenge \| Lady X Search DVinfo.net for quick answers \| Buy from the best: DVinfo.net sponsors

December 23rd, 2003, 09:25 AM	#3
Peter Moore Trustee Join Date: Oct 2002 Location: Arlington VA Posts: 1,034	Hey Adrian, Nope, it's NTFS. I long ago learned about the file size limitations of FAT32, so all my drives are NTFS. This is a programming issue. The files physically get larger than 2 GB, but nothing will play them properly. I got the AVI file spec from MSDN. Making uncompressed AVIs (which I'll then compress with Vegas) shouldn't be too hard.

December 23rd, 2003, 10:21 AM	#5
Peter Moore Trustee Join Date: Oct 2002 Location: Arlington VA Posts: 1,034	Ah, then that's almost certainly the problem. The VFW SDK probably is only creating Type 1 files, not Type 2, but I believe the AVI specs I got from Microsoft explain how to do type 2. Or I can probably find it one way or another. Outputting multiple files is a great idea, which I can most certainly program my Matrix raining code gemerator to do. Thanks for the suggestion!

December 23rd, 2003, 10:24 AM	#6
Colin Browell Regular Crew Join Date: Dec 2003 Location: UK Posts: 55	It does indeed sound like the signed int problem that Peter is describing. AVIs are RIFF formatted with (originally) a single RIFF section occupying the whole file. The RIFF header contains a 32 bit length field which will only allow files up to approx 2 GB. Anyway the format was extended (Open-DML standard) to allow multiple RIFF sections and therefore allow a MUCH bigger file size. The VFW AVI functions do not use the Open-DML extension and so are stuck with a maximum file size of 2GB. I am using DirectShow which does not have this limitation. If you need to stick with VFW then you may be able to find some OpenDML AVI parsing code somewhere on the net - in VirtualDub for example. Rob, I'm not sure what you mean but I think that the type 2 AVI file that you are referring to is the Open-DML standard (rather the the DV type 1 or 2)? If so it should really be referred to as an OpenDML AVI to avoid confusion with the DV types which are unrelated.

December 23rd, 2003, 12:26 PM	#7
Peter Moore Trustee Join Date: Oct 2002 Location: Arlington VA Posts: 1,034	Hm, interesting. I am actually not using DV, but uncompressed hi-def (otherwise this wouldn't be an issue). That's good to know DirectShow doesn't have this problem. Maybe I'll just use that, should be easy enough. Thanks a lot.

December 23rd, 2003, 03:30 PM	#10
Peter Moore Trustee Join Date: Oct 2002 Location: Arlington VA Posts: 1,034	Colin, All IU'm doing is generating bitmaps in memory (actually DirectDraw surfaces, because they're fast, but all the same) and turning them into movie frames. It's extremely easy to do that with VFW - seems straightforward enough. Any suggestions for approaching that problem? Rob, Yes I read the OpenDML spec and that's exactly what AVI "Type 2" is as MS calls it. Basically all they do is have multiple RIFF chunks in the same file, each of <= 1 GB, so that a legacy player can read the 1st GB worth and advanced players can read the whole thing.

December 23rd, 2003, 04:41 PM	#11
Colin Browell Regular Crew Join Date: Dec 2003 Location: UK Posts: 55	Ah yes, there's quite a good explanation at http://groups.google.com/groups?hl=e....freeserve.net Sounds like your app may be quite simple then. You may just have to write a source filter and feed it into a file writer.