An open source method for exporting back to the JY-HD10 or DVHS

[David Newman]

Paul,

The camera happily stores and plays back GOP lengths of 15 frames, so don't let that be a concern. Plus 15 is better for image quality. Mux-rate = is the transport stream bit-rate, i.e. with everything muxed together (audio and video) into 188 byte transport stream packets. I'm not sure what this would mean for using mpeg2enc. Bitrate is confusing as Mb/s could be a 1024 or 1000 multiple, than why I said approximately 19.2Mb/s. I don't remember the camera exact bit rate. Within the Premiere exporter we use 19400kb/s for the default mux-rate, although we can support wide range.

[Peter Robert]

Hi David,

DVHSTool doesn't work neither. It can export the ts file to jvc d-vhs. I can see the picture and hear the audio which is ok. But when I rewind the tape and play back again, no audio and video is a mess.

[Paul St. Denis]

Thanks for the information David, I'll try 15 frame GOP's in mpeg2enc tomorrow.

Peter, you might have found your solution already using DVHSTool as David suggested. I tried to find a win32 port for mpeg2enc today, I could find the binary as part of VCDEasy, but not the source code, the source code is availble for the unix version, but I couldn't get it to compile in VC++ or cygwin.

Alex Raskin has a great thread describing his workflow for going back to the camera on this forum from Premiere.
http://www.dvinfo.net/conf/showthrea...ghlight=womble

I used the same steps starting from "Then use TMPGENC to convert that AVI Movie into ES..." after exporting an AVI and AC3 from Vegas.

One minor typo, Alex suggested using the "hyffyuv" codec which should read huffyuv, available at http://neuron2.net/www.math.berkeley...g/huffyuv.html

tempgenc is available http://www.tmpgenc.net/e_main.html

DVHStool http://kgbird.com/DVHSTool/

womble http://www.womble.com/index.shtml

[Paul Mogg]

Wow, Paul this is wonderful what you have done, congratulations on your efforts! I just came back from the UK where I did quite a bit of filming with my HD1OU and now finally I'll be able to edit and archive back to DVHS on the Mac, thank you so much.
Just out of interest, I don't know if you saw my previous postings on the changes I was told to make to Mpeg2enc ( by the ffmpegx forum guys) to get it to encode HD resolutions, but I was wondering if it required more extesive changes than just changing those few parameters and re-compiling? You might want to inform Major over at the ffmpegx forum of what you've been able to do as he had it on his to-do list for the next version release to incorperate HD encoding under Mpeg2enc, and it sounds like you've just saved him a lot of work.

Thanks again.

[Paul St. Denis]

Thanks for your congratulations Paul, it is nice to hear.

The solution for mpeg2enc that I found was very simple although
I am sure it is not the way the developers of mpeg2enc would have done it.

I could not encode at bit rates higher than the 15Mbs despite having the --no-constraints parameter, so I just removed the check for the bit-rate. This would affect people who want to make sure their bit-rates conform to specifications for VCD,DVD etc.

The result is a program which will work fine for our needs is no longer general purpose.

I am focusing on getting the steps to be more user friendly, I should be able to post the results in a week or so.

I wish the creator of ffmpegX who uses open source tools would in turn make his work open source.

[Heath McKnight]

<<<-- Originally posted by Paul St. Denis : I wish the creator of ffmpegX who uses open source tools would in turn make his work open source. -->>>

I thought that was a rule for open source...

heath

[Paul St. Denis]

I think that depends on if ffmpegX can be considered a single program or a group of programs that work alongside each other, here is a quote from the GPL FAQ

... in many cases you can distribute the GPL-covered software alongside your proprietary system. To do this validly, you must make sure that the free and non-free programs communicate at arms length, that they are not combined in a way that would make them effectively a single program.

http://www.gnu.org/licenses/gpl-faq....prietarySystem

I think the "arms length" argument could be made for ffmpegx, it is not combined with the open source programs but runs on its own and calls these programs when ever they are needed.

[Paul St. Denis]

Paul,
I was just thinking that it should be possible to use ffmpegX to convert from a HD Quicktime file to program stream if ffmpegX's version of mpeg2enc is replaced with the version I posted above.
I will not be able to try this until Monday.
By the way thanks for all the great posts you've made in the last few months, I was just checking out some of the older threads and I see you had a way to edit in Final Cut last May! I didn't think the camera was even available then.
I also noticed that Steve Mullen used to post here quite a bit, where did he go?

[Paul Mogg]

Thanks, I'll have to try putting your Mpeg2enc version in ffmpegx and see how it works. Should I theoretically then be able to encode from any other quicktime format and not have to go through the MJPEG convertion first? That would be great.


Cheers.

[Paul Mogg]

Hey Paul, I just tried your version of mpeg2enc in ffmpegx, it failed unfortunetly, but here's the terminal window output showing the error. Let me know if I specified someting incorrectly or if it's an easy fix. Thanks

Encoding started on Sun Jan 11 10:31:29 PST 2004
INFO: [yuvscaler] yuvscaler (version 1.7.0) is a general scaling utility for yuv frames
INFO: [yuvscaler] (C) 2001-2002 Xavier Biquard <xbiquard@free.fr>
INFO: [yuvscaler] yuvscaler -h for help, or man yuvscaler
INFO: [mpeg2enc] SETTING AltiVec for MOTION!
INFO: [mpeg2enc] SETTING AltiVec for TRANSFORM!
INFO: [mpeg2enc] SETTING AltiVec for PREDICTION!
INFO: [yuvscaler] input: frame size: 1280x720 pixels (1382400 bytes)
INFO: [yuvscaler] input: frame rate: 30000/1001 fps (~29.970030)
INFO: [yuvscaler] input: interlace: none/progressive
INFO: [yuvscaler] input: sample aspect ratio: 1:1
INFO: [yuvscaler] from 1280x720, take 1280x720+0+0, NOT_INTERLACED/PROGRESSIVE
INFO: [yuvscaler] scale to 1280x720, 1280x720 being displayed, NOT_INTERLACED/PROGRESSIVE
INFO: [yuvscaler] Scaling uses the RESAMPLE algorithm,
INFO: [yuvscaler] without line switching
INFO: [yuvscaler] without time forwarding
INFO: [yuvscaler] Without luminance correction
INFO: [yuvscaler] Without chrominance correction
INFO: [yuvscaler] frame rate: 29.970 fps
INFO: [yuvscaler] Scaling ratio for width is 1 to 1
INFO: [yuvscaler] and is 1 to 1 for height
INFO: [yuvscaler] Specific downscaling routing number 4
INFO: [yuvscaler] output: frame size: 1280x720 pixels (1382400 bytes)
INFO: [yuvscaler] output: frame rate: 30000/1001 fps (~29.970030)
INFO: [yuvscaler] output: interlace: none/progressive
INFO: [yuvscaler] output: sample aspect ratio: 1:1
INFO: [mpeg2enc] Selecting generic MPEG2 output profile
INFO: [mpeg2enc] Assuming norm NTSC
INFO: [mpeg2enc] Encoding MPEG-2 video to /Volumes/SATA RAID 1/HD Testing/test1_mpeg.m2v
INFO: [mpeg2enc] Horizontal size: 1280 pel
INFO: [mpeg2enc] Vertical size: 720 pel
INFO: [mpeg2enc] Aspect ratio code: 3 = 16:9 display
INFO: [mpeg2enc] Frame rate code: 4 = 30000.0/1001.0 (NTSC VIDEO)
INFO: [mpeg2enc] Bitrate: 10000 KBit/s
INFO: [mpeg2enc] Quality factor: 5 (Quantisation = 5) (1=best, 31=worst)
INFO: [mpeg2enc] Field order for input: none/progressive
INFO: [mpeg2enc] New Sequence every 9999 Mbytes
INFO: [mpeg2enc] Assuming non-video stream of 422 Kbps
INFO: [mpeg2enc] Search radius: 16
INFO: [mpeg2enc] GOP SIZE RANGE 9 TO 15
INFO: [mpeg2enc] Setting colour/gamma parameters to "NTSC"
INFO: [mpeg2enc] Progressive format frames = 1
**ERROR: [mpeg2enc] Horizontal size is greater than permitted in specified Level

[Paul St. Denis]

Ok I can compile a version that ignores the size constraints.
In the mean time is there any way to pass the --no-constraints flag to mpeg2enc from ffmpegX?

[Paul St. Denis]

Paul, can you give this one a try?
http://www.celt.sunysb.edu/paul/mpeg2enc_HDTV.zip

[Paul Mogg]

Thanks Paul, just to let you know, I also tried your command line method but for some reason I get the response "command not found" for ffmpeg, even though I've checked that the ffmpeg program file definately is in the directory I'm in (usr/local/bin) using "ls", any ideas? Also FYI the input file I was using for the ffmpegx attempt was an 8 bit uncompressed HD file.
...Also I can't see any way to implement the "no constraints" flag from within ffmpegx.

...FYI, below were the instructions that Major (the guy that wrote ffmpegx) gave me to solve this problem, I see that he talks abou a "Level ID" needing to be set to 4, so this might be clue also.

Good luck, I can't wait to try it out!


well im a sucker for punishment -- i downloaded all the source code used in FFmpegXrelating to mpeg encoding and using the FFmpeg component of FFmpegX , no (does do transport stream though - but max resolution is PAL DVD)

but -

mpeg2enc - YES !

you (or someone) will have set the following:



set to 4



Code:

/* Level ID */

Specifies coded parameter constraints, such as bitrate, sample rate, and maximum allowed motion vector range.

Code Meaning Typical use ---- --------------- ----------------------------------------------- 4 High Level HDTV production rates: e.g. 1920 x 1080 x 30 Hz 6 High 1440 Level HDTV consumer rates: e.g. 1440 x 960 x 30 Hz 8 Main Level CCIR 601 rates: e.g. 720 x 480 x 30 Hz 10 Low Level SIF video rate: e.g. 352 x 240 x 30 Hz




set to 512 (for 19.2meg/s standard 720p)



Code:

/* vbv_buffer_size (in multiples 16 kbit) */

Specifies, according to the Video Buffering Verifier decoder model, the size of the bitstream input buffer required in downstream decoders in order for the sequence to be decoded without underflows or or overflows. You probably will wish to leave this value at 112 for MPEG-2 Main Profile at Main Level, and 20 for Constrained Parameters Bitstreams MPEG-1.



set to 2 for DC_P



Code:

/* intra_dc_precision */

Specifies the effective precision of the DC coefficient in MPEG-2 intra coded macroblocks. 10-bits usually achieves quality saturation.

Code Meaning ---- ----------------- 0 8 bit 1 9 bit 2 10 bit 3 11 bit



set to all 24's (or 32's) except for position 1 which must be a 8 (you will need to build a table they explain (simple))




Code:

/* name of intra quant matrix file ("-": default matrix) */

Setting this to a value other than - specifies a file containing a custom intra quantization matrix to be used instead of the default matrix specified in ISO/IEC 13818-2 and 11172-2. This file has to contain 64 integer values (range 1...255) separated by white space (blank, tab, or newline), one corresponding to each of the 64 DCT coefficients. They are ordered line by line, i.e. v-u frequency matrix order (not by the zig-zag pattern used for transmission). The file intra.mat contains the default matrix as a starting point for customization. It is neither necessary or recommended to specify the default matrix explicitly.

Large values correspond to coarse quantization and consequently more noise at that particular spatial frequency.

For the intra quantization matrix, the first value in the file (DC value) is ignored. Use the parameter intra_dc_precision (see below) to define the quantization of the DC value.





use default (all 16's):




Code:

/* name of non intra quant matrix file ("-": default matrix) */

This parameter field follows the same rules as described for the above intra quant matrix parameter, but specifies the file for the NON-INTRA coded (predicted / interpolated) blocks. In this case the first coefficient of the matrix is NOT ignored.

The default matrix uses a constant value of 16 for all 64 coefficients. (a flat matrix is thought to statistically minimize mean square error). The file inter.mat contains an alternate matrix, used in the MPEG-2 test model.

[Paul St. Denis]

/usr/local/bin needs to be in your search path, I am surprised it isn't already as I thought it would be by default. You can get around this by replacing ffmpeg with /usr/local/bin/ffmpeg in the command. You will probably need to do the same for mpeg2enc.

Typing "echo $PATH" will let you know what your search path is.
Sorry anout the hassle and thanks for trying it out.

This might work in ffmpegX
http://www.celt.sunysb.edu/paul/mpeg2enc_HDTV.zip

[Paul Mogg]

IT WORKS!! ...and whats more, it works fabulously! I've tried encoding an uncompressed 8 bit HD movie, a Pixlet codec movie, and a DC30 codec movie, and all worked perfectly. The Pixlet movie was about 4 minutes long.
The quality is also very high from what I can judge so far, (which is amazing in itself) no obvious artifacting introduced, nor any gamma shift as I've seen occur on some "professional" encoders. I've only got the Apple SD MPEG2 encoder to compare it with, and I would say it does an equal or better job than that, but I'll let others judge. This is definately good enough for me.
An additional plus is that the Apple Quicktime player will play these HD movies on your desktop.
I haven't tried the conversion to Transport Stream or going back to the camera or DVHS deck yet, but I'll let you know.
It would seem you've achieved the Holy Grail here Paul, a very low cost ($15 for FFmpegx shareware), high quality, and very reliable HD MPEG2 encoding method.
We now have a complete and very flexible editing pathway for the Mac for HD1OU footage.
My hat is off to you and your associates!!
You should make a separate thread with a big announcement about this.
Below I've documented the settings I found tha seem to work for Ffmpegx, for people to use, you may have better reccomendations of course.



FFmpegx program interface settings for compressing a Quicktime HD movie (any codec 1280*720) to an MPEG2 HD (1280*720) Program Stream.

Open FFmegx

Use the "Open" and "Save As" tabs to select your input Quicktime HD Movie (1280*720), and output file name for the HD Mpeg2 file that will be created, (you can also drag and drop to input the file name).

From the "Video Codec" pull-down menu select "MPEG2[.MPG](mpeg2enc)"

Check the "Encode video" box

In the Bitrate calculator "Video Bitrate" box enter 10000

In the "Video parameters" section enter 1280*720 as the sizes in the "Video Size" boxes

From the "Autosize" drop-down menu select "16:9"

From the "Framerate" drop-down menu select "NTSC (29.97)"

Click on the "Audio" tab

From the "Audio Codec" drop-down menu select "MP2"

From the "Audio bitrate" drop-down menu select "384"

From the "Sampling" drop-down menu select "48000"

From the "Channels" drop-down menu select "Stereo"

From the "Mode" drop-down menu select "CBR"

Click on the "Filters" tab

Make sure that no filters are selected (unless there's something you specifically want to do here)

Click on the "Options " tab

In the "mpeg2enc options" section make sure that ONLY the following boxes are checked, un-check all others:
"GOP headers"
"Altivec" (assuming you're using a G4 or G5 Mac
"Decode wit Quicktime"

From the "Profile" drop-down menu select "Generic MPEG-2"

From the "Q Matrix" drop-down menu select "Standard"

From the "Interlace" drop-down menu select "Not interlaced"

From the "Thread" drop-down menu select "Dual CPU" if your computer has dual processors, or else select "Normal"

In the VBV size box enter "400"

Make sure the "Constant bitrate" box is un-checked

In the Q min box enter "5"

Click on the "Tools" tab

Make sure the "Keep elementary streams" tab is unchecked

Hit the "Encode" button and off it will go.