Ok a screengrab will do..let me get to work...i will post one then...the last issue we now have is the save of raw data..we are waiting on a 3rd party at the moment for a software fix of the profiler we are using...once this is done we should have a working beta of CineLink!

www.dv3productions.com/pub/dog.tif

shot with an OLD 16mm c-mount lens ;)

Hey Obin, some great lattitude and tonal range in that frame. However, I noticed a strange grid-like pattern when viewed at larger magnification.

yes grid..we have the most basic bayer filter you can have..this is why you see the grid. If your digital still camera took every pixel from the chip and gave it an RGB value it would look the same.

after we get raw save working we will have some Bayer Filter options

That's a huge file for one frame. I can see the grid, but besides that, it has very film like detail and tones. OK great...so now we demand a second image.

No, really it looks great.

found a workaround for the profiler issue..back to work..more later

Appreciation
 

Greetings all. I'm a newcomer, joined years ago but rarely kept up with DVi... but I am exceedingly pleased to come back and find amazing projects like this going on. Obin, you are da man. I have no idea who you are but your perseverence with this project is amazing. 175 pages of replies? What in heaven's name?! And this is soooo the right way to do things, too - start with a sensor, not a Nikon F and a CD/sandpaper.

I will be enthusiastically checking up on this from day to day.... the passion that a lot of folks are displaying for DIY cinema cameras (sic) is great, but here is one place where the passion has some real direction to it.

Also, Obin, if you need any help with very accurate machining (lathing or CNC milling) let me know. Also, if you have any parts that need to be physically aligned to very accurate tolerances, give me a holler - I'm a flutemaker and we have some amazingly accurate ruby-probe 3D measurement tools here in the shop (+/- 0.0001").... I can build you fixturing or whatever you need, at cost. All the equipment belongs to my father and I (old family business) so I'm open to doing whatever work you need to get this camera to the place where you want it. No idea if you'd ever need this sort of work, but I figured if nothing else you'd appreciate knowing that there's someone else willing to donate some time to this project. Our shop is very flexible so don't be afraid to email me (klimermonk -at- hotmail -dot- com) if you have something in mind.

JL

John;

thanks a bunch for the offer..i have a small cnc machine but it seems you have years of working with them on your side(I don't) I will take you up on the offer when the time comes...or atleast you could help if we want to do it at our shop? I am sure good things will come however we do it.


thanks for the positive feedback... ;)

we have some good news.

today we ran save tests that are showing we can get full 24fps 1080p datarate on our 2 drives. We have a test app that saved 4 gigs in 512kb chunks on the disks at 97MB/Sec with no problems...now it's time to write that in the code and get some real progress....

.

Hi Obin,

Congratulations! Just one thing I noticed...You should only need about 72MB/sec for 24p, 12-bits per pixel. Are you receiving one pixel in 16-bits? With the GigeLink I can pack my 10-bit data, two pixels in 3 bytes. The 10-bit data is bit-shifted by two ("normalized 12-bit").
This packing results (for me) in only moderate overhead for the display (more shifting by 4 must be done).
I'm assuming your capturing about 48MP/sec.

we don't have pixel packing with CameraLink

Kyle, what is your project? are you working on software?

Obin,
> we don't have pixel packing with CameraLink
Then you are unfortunately stuck with the 16-bit pixels. However, you might try a test: Do the 16-bit to 12-bit conversion in software. The CPU overhead should not be too bad: A couple of shifts and an add in C/C++, and you get 72MB/sec. Your programmer sounds very experienced. 10-bit packing (4 pixels in five bytes, 60MB/sec) is also a possibility, but that increases the bit-twiddling. Yet another possibility: "10.667-bit" packing -- 3 pixels in 4 bytes (64MB/sec).

> Kyle, what is your project? are you working on software?
I am pretty much finished with the software. It is merely a grabber application, like yours, with capture, display and recording to disk. For one Gigelink interface (with SI-3300RGB or SI-1300M), or two cameras (stereo, 2x SI-1300).
It all works fine. There is also a converter dialog, which exports image sequences from the RAW file (BMP (24-bit), SGI (48-bit), and DPX (log 30-bit).

You tend to get married to whatever grabber SDK you use. I lucked out with the GigeLink: It is well-written and well-documented.
Most of the CL boards I looked at last summer do perform pixel packing (BitFlow, Coreco, Leutron) --- not that that helps you.

Kyle are you using 10-12bit in your software? or are you using 8bit captures? what are you going to do with your software?

> are you using 10-12bit in your software?
Application is hot-rodded for 12-bit packed. No 8-bit captures (but simple enough to implement). Device can be 10-bit (3300, 1300) or 12-bit.
> what are you going to do with your software?
I shoot with it almost every day. And a company is involved.

Can you keep us updated, and give details when the time comes?

Wayne.

Kyle what camera are you using?
do you have any clips I could see?
are you displaying in black and white or color?

thx for the info on packing we are playing with that ...

Obin & Wayne,
I'll let you all know when the software goes public, or not.

> thx for the info on packing we are playing with that ...
cool. this means also, as you might guess, more work at the RAW conversion end.

> Kyle what camera are you using?

Two mono SI-1300's, one Bayer SI-3300.

> are you displaying in black and white or color?

both, depending on the camera. color display is quick and dirty. stereo display is red/cyan.

My clips are large, I am still learning, and I am a bit shy about posting clips. Right now I am mainly using the SI-1300's for stereo work. I could post a 16-bit linear mono still picture some place (about 2 MB), if anybody is interested. (where? login?).

www.uploadhouse.com or www.imageshack.us

no luck uploading picture to upload house: not allowed for for PSD, SGI or TIF. Limit at imageshack is 1MB. If anyone wants the 1.4 MB (16-bit mono!) TIF file, just email me.

http://img216.exs.cx/my.php?loc=img216&image=vasesample9lt.png

I cropped it. This is un-gamma corrected, un-color-corrected. I don't know if the 10-bit data in the 16-bit TIF survived, or if it's just a plain 8-bit image. Looking at an uncorrected 8-bit image would be a waste of time.

wooohooo..and who says digital is not as good as film??


www.dv3productions.com/pub/BAND.jpg

a photo from a photoshoot I just did...like? hate?

using cheapo Sigma digital lens on my Canon 10D 6megapixel with some NICE light ;) a bit of photoshop work on the curves and the contrast...

I LOVE cmos chips...so "organic" I really see CCD and CMOS now when I look at professional work still/motion pictures

that looks tight obin. now we just need 24fps :)

yes 24fps....that is the ISSUE at hand...LOL

making more progress on save...Kyle we are working with the bit packing...looks like it may not save enough overhead/datarate to be worth it...

Obin,
Sorry to hear the packing is not helping you with your problem. I was thinking that the 64MB/sec packing would be the best (packing 3 10-bit pixels into one DWORD). Read three DWORDS (6 pixels) and write two DWORDS. That puts additional bus bandwidth at (96 + 64) MB/sec. If you have a 533 MHz FSB (2133 MBsec), I thought it just might work. The CPU should not be overloaded -- it sounds like a memory bottleneck. Oh well...

I am finally making some slow progress. I've spent the morning
developing a more efficient way of pacling bits. I'm in the process of
validating the code right now to make sure I am not losing any data. This
will take a bit as things are totally unintuitive and I have to check the
bits every few seconds to make sure things works properly. The main logic is
that pairs of pixels that are in short integes (16bit) are combined together
in 3 bytes thus saving 25%

of the storage requirements. The only strange
going on so far is that the order of these 3 byte packets are flipped in the
conversion for some strange reason. It seems to be happening during the
copying process. I'm debuggin this right now, but in any case it might be
irrelevant if they get flipped back during conversion back to 16 bit format
to create the TIFFs

I wan to try the bytes to Integer conversion first to
make sure we do no lose any data on the trip back. I got some code this
morning from that friend, but I cannot compile it as he forgot to send some
header files with it. It looks like it may be even more efficient, but he is
in meetings all day so I will not get the headers unitl tonight at the
soonest.

Obin,
You'll get it to work, I am sure! Two more points...
1) I wouldn't worry too much yet about doing it "perfrectly correct", and confirming the unpacking during the export process. More important is "optimally", i.e., the inner loop code. You want to make sure that your performance is not compromised, and that you are truly gaining something by writing less to disk, which in your case will be 72MB/sec
2) I have recorded about half a terabyte, so far. I just want to add that every byte you save is a byte earned: in the conversion process, transfering files, and archiving. It will be worth it.
My two cents.

That wont work as the pixels coming out of the SDK are 12bit even
though the underlying data is 10bit. You would need to convert them from 12
to 10bit and that would waste too many cycles. If they were already 10bit
this would be very easy to implement and would be just as fast as the 12 bit
solution with much better performance. I will fiddle with the SDK a bit
later to see if it is possible to get the actual 10bit pixels from the
framebuffer. Since Xcap does not give that choice I doubt it.

> That wont work as the pixels coming out of the SDK are 12bit even though the underlying data is 10bit.

you bit structure is then thus?

msb lsb
0000bbbb bbbbbb00

> and that would waste too many cycles.

not necessarily. this depends on how you are masking and shifting.

but if the 12-bit packing works for you, then great!

Right now I am testing with packing 2 pixels into 3 bytes thus
saving 25%, but I do that with a single shift and masking. To do 10 bit
packing I need to align into 8bit boundaries as you can only copy memory
fast in 8bit chunks and I would have to pack 5 pixels in 6 bytes, as doing
otherwise would leave dead space which would defeat the purpose of it all.
This would mean some more shifts, but you would save 38% space instead of
25%. In any case I am testing data integrity now, as it is no use going very
fast if the data gets corrupt in the first place while recording. I will
write code to try 10bit packing too and then I will profile it to see if I
gain enough speed from the extra work to make the space saving effective.

Regards,

Luc

George Lucas will be re-issuing all the Star Wars films in 3D, one film per year, starting in 2007.

Kyle ?


"I'm a man on a mission when it comes to 3-D," Cameron said. "I will be making all of my films in 3-D in the future. We need exhibition to come in to own a big chunk of the (emerging 3-D) market."



hmm..Kyle works for a "firm" this firm or "group" wants "3d imaging" kyle gets on the message board and NEVER tells what he is really doing.....makes you wonder huh...

rock on Kyle! I wish you the best of luck... someday you can tell me what your ---really doing--- ;)

I know this is probably not the best place for code, but here goes anyway....

// packing three 10-bit pixels, received as shorts,
// into DWORD
// assuming 10-bits here:
// 0000bbbb bbbbbb00
// savings of 33% over unpacked 16-bit format

unsigned short *src;
unsigned long *dst;

*dst = (unsigned long) *src++;
// dst = 00000000 00000000 00001111 11111100

*dst += (((unsigned long) (*src))<<10);
src++;
// dst = 00000000 00222222 22221111 11111100

*dst += (((unsigned long) (*src))<<20);
src++;
// dst = 33333333 33222222 22221111 11111100

dst++;


[ code is not tested, nor compiled...but no byte arithmetic is done...just longs...no masking...just adds...luc or you can email me, obin ]

>> hmm..Kyle works for a "firm" this firm
>> or "group" wants "3d imaging"

heh, heh...thanks, Obin.

Well, i used to live down the street from George in San Rafael. Now I live down the street from Arri.

However, the reality is much less exciting, since I'm just doing what all of you are doing....

WORKING ON MY SCRIPT!!!

But it's nice to see Jimbo and George "möge die Macht bei Dir sein" Lucas getting with the program. It makes me feel a little less insane.

<<<-- Originally posted by Obin Olson : George Lucas will be re-issuing all the Star Wars films in 3D, one film per year, starting in 2007. -->>>

I wish he get over it, maybe even film the last three episodes. I am fed up with the reruns everytime he releases a money spinning modification of the original (which looked better than the last mod anyway).

Maybe they can replace princess Lela with a better looking computer generated character this time, and the dorky cloths, and replace Mark Hamil with a computer generated young Brad Pit, or Matt Daemon (the female audience would love that ;). You think I'm joking, eventually this is what they will be able to do. actually they could give Luke and Hans a real challenge and replace Lela with Julia Roberts, or freak everybody out by replacing her with Meg Ryan (no Jeniffer Lopez).

Actually I can almost feel this post being sucked away by the local Rob Star.

Anyway, Obin, do you program, I thought you had a programmer doing all of this for you?

Packing data
 

Obin,
This might be the time do dust off some assembler skills. At least, look at the code generated by the C++ compiler. I haven't checked the instruction set on an x86 in years but I think that there was a barrel shifter that could do any shift in a single cycle. Sometimes the compilers don't pick up on these things and do 4 separate shifts.

The only place code really needs to be optimized is in the inner-most loops - the stuff you do a million times an image - packing is certainly one of them.

One trick I learned back when coding was a simpler place was to write a function in C and compile everything. Then I could change the function at the assembly level and know all my stack and data was correct.

Of course all this could just be showing how obsolete my programming skills have become.......

Hi Obin, I posted this over on the Andromeda forums.

The source code for the DNxHD codec is freely available for download from:

http://www.avid.com/forms/DNxHDinfo.asp

I've already tested the 10-bit codec and the results were very promising.

I am glad to hear that Brad as I am going to use that codec once we get the save working in CIneLink...

But this is 8 and 10 bit YUV only right?


<<<-- Originally posted by Brad Abrahams : Hi Obin, I posted this over on the Andromeda forums.

The source code for the DNxHD codec is freely available for download from:

http://www.avid.com/forms/DNxHDinfo.asp

I've already tested the 10-bit codec and the results were very promising. -->>>

You guys realise that DNXHD 10-bit is 220Mb/s which is close to lossless compressed bayer 1080p?