Linux Pipeline (in the works)

[Chris Cherrett]

//////////////////////////////////////////////////////////////////////////////////////////////////
Summary: Command used to capture/monitor live video in Linux
//////////////////////////////////////////////////////////////////////////////////////////////////
To Monitor:
mpg1394grab | mpeg2dec -t 4096

To Capture:
mpg1394grab > testfile.mpg
//////////////////////////////////////////////////////////////////////////////////////////////////

I decided to write this thread based on my frustrations from this thread:

http://www.dvinfo.net/conf/showthread.php?t=97953

I will work on this thread as time goes on in order to provide the best most up-to-date information available.

I have been working on getting Linux working with my JVC GY-HD100 for about a month or so now. I have had some good success today and thought I would start a thread to share it. I use 100% Open Source software for all my graphics work (raster,vector,3D,flash). For the most part I am just getting video in Linux and find it a fantastic journey!

Software Used: (this is not a full list yet but I will get into that soon)

Blender
Cinelerra
mpginfo
mpg1394grab
mpeg2dec
mplayer
vlc
gimp

For now I am not going to talk about NLE or Compositing but that will come soon!

(place holder for NLE and compositing talk)

While trying to figure out a low cost solution to a focus field monitor for my camera I stumbled upon a few good solutions for getting HDV video to write to my hard drive and use my laptop as a reference monitor.

The solutions were not the easiest and HDV is a highly uncharted venture in Linux but it is possible with 100% open source software.

1. Capturing Video:

I found this program called mpg1394grab at the following url:

http://www.kinodv.org/filemanager/do.../mpg1394grab.c

It needed to be compiled with the gcc compiler which is in most Linux distributions by default.

the program is simple. It looks to your /dev/raw1394 device (firewire device on linux) for a mpeg2 stream. If it sees one then it writes it to disk.

mpg1394grab > somempegfile.mpg

will start the program listening to the /dev/raw1394 firewire device on your computer. When you hit REC on your camera (or PLAY if you are in VTR mode) mpg1392mpg will start to write the file to your computer.

2. Monitoring (NOTE: this solution has a .5 second delay on my p4 2.4, 1GB RAM. I am sure that would be different with newer hardware.)

The best option I have found for monitoring so far is to use the mpg1394grab program and pipe ( this symbol -> |, pipe is shift+backslash) to mpeg2dec. mpeg2dec is a simple comandline program that can take a mpeg2 stream and play it back for live monitoring.

The command is like this:

mpg1394grab | mpeg2dec -t 4096

At this point you should be seeing the output from your camera.

A few words about mpeg2dec:

- the t option states:
-t use transport stream demultiplexer, pid 0x10-0x1ffe

I don't fully understand this yet but after some digging I ran a program called mpginfo like this on an existing file and found some information:

mpginfo testfile.mpg

this outputted the line:

Stream 3: MPEG 2 video [pid: 4096]

this is the whole output:

Skipped -2 zeroes at start of file
mpgtx: AT EOF - please stop me!
mmm, this file does not start with a pack, offset: -2
use the desperate_mode switch as the first option -X to search for a header in the whole file!
if you want to force the operation. May yield to an endless loop if no valid header is found!
Does not even begin with a 00 00 01 xx sequence!

No success at all.
testfile.mpg
Mpeg 2 Transport Stream [1 program]
Program N° 1 contains 3 Elementary Streams:
maybe scrambled (CA ID 65535)
1 additional descriptor was not handled
Stream 1: User Private data [pid: 3842]
Stream 2: MPEG 1 audio [pid: 4098]
1 additional descriptor was not handled
Stream 3: MPEG 2 video [pid: 4096]
video version 58
1 additional descriptor was not handled

from there I was able to use the PID for the t option and I could view my stream live!

3. Capture and Monitor at the same time:

The easiest way I found was to run the command for monitoring in one terminal window:

mpg1394grab > testfile.mpg

and run the capture command in another window:

mpg1394grab | mpeg2dec -t 409

Now to get a huge external hard drive for the computer and all is well! :)

[Stephan Ahonen]

Unfortunately this post is exactly why Linux isn't ready for primetime outside of the web server market. As much as I agree with the philosophy, I simply don't have the time or energy to be futzing around with technology when I've got deadlines to meet and clients to satisfy. What you're paying for with commercial products is the ability to just sit down and get things done.

[Drew Curran]

Chris

Its people like you who help keep things moving forward, and give people platforms for new ideas to spring from.

Thanks for posting this.


Drew

[Chris Cherrett]

Stephan,

I agree that Linux is not ready for prime time HDV work. As for DV the tools are built into the NLE's already. I posted the thread to show the possibility of what can be done at this point. My hope is that when someone that wants to run Linux will only have to run the command:

To Monitor:
mpg1394grab | mpeg2dec -t 4096

To Capture:
mpg1394grab > testfile.mpg

This thread is being linked to from:

http://www.pipapo.org/pipawiki/Cinelerra3/BrainDump

which is the beginning talks towards the next generation Cinelerra NLE. There is the possibility that what I can come up with here and document will become a monitoring feature in Cinelerra as well as the method for capture direct to the NLE.

So that said you are right but only for a very short time. :)

Drew,

It is my pleasure :)

I am a programmer by trade and love to see Linux move forward. I have seen Linux make major moves in every area over the past 7 years. It only takes a few interested developers and Linux starts to shine. Just last month I was looking to purchase a new rendering package to deal with SSS in blender. That same day I found that a developer had been working on integrating it into blender. In two weeks time blender had SSS built into the internal renderer. Things happen fast in Linux :)

The best thing about Linux is I am not locked into an OS jail. I am free to make my computer do what I need it to. I can build specialty embedded devices to strap to my camera that to buy would cost thousands of dollars and do next to nothing for me. I look around at devices being sold for video and think "Man I could build that for $25-$200 dollars". Then I can post my results to the web and others can join in and help to build it as well.

[Chris Cherrett]

Stephan,

I needed to speak to the statement you made:

"Unfortunately this post is exactly why Linux isn't ready for primetime outside of the web server market"

This comment is badly outdated now. It use to be true but not any more. I install Linux for all sorts of people for all sorts of reasons. Linux has many many mature software solutions that make it a very attractive choice for fast,secure and free computing. HDV is just being developed in Linux now but in 6 months to a year you will no longer be able to say that Linux is not good for HDV editing. As for Hollywood production, Linux is king in a lot of areas.

[Stephan Ahonen]

Every single experience I've had with Linux, even fairly recently, has been that it's all "by geeks for geeks." Hey, I'm a geek too. Geeks love experimentation and building new things and Linux is a great way to do that. But when I have a client and a deadline, I don't have time for that, and I especially don't have the energy or sanity to deal with untested bleeding edge solutions that were built in some random person's spare time. If the choice is between spending hours hacking regex and delivering a product to my client, guess which one I'm choosing.

My last experience with Linux went something like this. After hours of digging through man pages and hacking the X config file, it STILL would not use my dual head rig fully, my second monitor was limited to 1024x768 when it was capable of 1280x1024. I tried installing Cinelerra for editing. It wouldn't run. No, literally, wouldn't run. I tried compiling from scratch and installing from packages. Didn't do anything. That's real impressive right there. But if it's anything like GIMP and Audicity, two other OSS products that Linux advocates love to point at when people mention the lack of media production tools for Linux, it's probably under-featured and has a terrible interface.

It's perhaps a little known fact that Blender was actually a commercial product before it went OSS. It was developed as an in-house rendering tool designed to get things done under pressure and deadlines, which is why it has such a wonderfully efficient interface. It's too bad the OSS community hasn't kept up with this philosophy.

[Chris Cherrett]

Stephan,

My intent on posting this thread was not to convince anyone that Linux was the best. My intent was to try to help the Linux goer from spending days researching this under developed area of Linux. It is part of what Linux is all about, community. I would love to be part of helping someone through a tough problem and sparing them the experience that you had.

When I run into problems in Linux the first thing I do is turn to the community. Through forums like this one and irc channels on servers such as freenode, you can obtain a wealth of knowledge from people that are very willing to give a hand.

I can understand your frustrations with Linux. I have felt them myself.

Stephan, I don't want a flame war with you. I just wanted to provide some help to the community. I am sorry you feel the way you do about Linux. As for the way X is configured there is work being done to eliminate people from having to configure it at all. I hope that soon Linux will be at the stage that you can try it again and have you suffer the frustrations.

P.S. If I had have know you at the time, I would have walked you through your problems you were having. Possibly I could help in the future.

[Steve Oakley]

well smoke, flint and flame run on linux... or don't they count ?

[Jiri Fiala]

Apparently Linux is moving forward in many areas, and latest distros are very user friendly even for non-geeks. But HDV is a very special area (hell even latest Final Cut 6 has big issues with ProHD), and linux devs cannot improve everything at once.

discreet linux apps are high-end sold with turnkey systems tailored for their particular tasks. You cannot expect any free distro to be able to do everything your 10 000 USD Windows/Apple rig does.

[Chris Cherrett]

Hey whatever you guys want to run is your own decision. At the end of the day I win contracts and people like my work. They never say "Hey was that done on a MAC or Windows". They don't care. I have not found an area in media where a client has known that I use all open source. It is very possible to produce professional quality media that can compete with what the best commercial apps produce. Again it is all in the artist as far as I am concerned.

I have been accepted as a graphic design vendor in several large organizations based on my work even though I broke their standards of MAC and Windows.

So at that if you want to use Linux and are having some issues let me know. In the end it is about the viewer enjoying your work, not what GUI you used. My tool set under Linux does not mess with my timelines because I know how they work. It would only mess you around if you did not know your way around and were depending on the application to get work out the door. But would that not apply to any app you did not know how to use? :)