so...

I have just reviewed things and we are withing a few ms of doing it
properly. If your machine was behaving the same as mine on the wait event we
would be there now. I will try a bunch of things and see how I can get
things to work properly. We are almost there. I'll keep you posted.


hmm..this is good...

whoooaaa

we have gone from 47% cpu overhead for preview alone - down to 2%!!!!!!!!!!!!!!!!!!

yes 2% not 20%!!!!

;)))))))))))))))))))

we have finally found out why the system was using so much CPU. The
problem was because of the tight internal loop that triggers the display and
capture. Without any processing I was gettting around 50% CPU use with it. I
am testing different ways of implementing this.


things are looking bright again....

Steve N. what MHZ does the Altasense chip run at to get nice rolling shutter free images? i forgot...is it 150mhz? can the Epix CL2 card deal with that?

THe weekend commercial shoot went well ( shot from 9pm till 4:30am ;) ) but the budget called for MINIDV ..arrgghhhhhh I HATE mini!!! compression is SOOO bad on that codec...and the DOF is like - um - well lets jsut say the whole frame has a very good focus on it...mostly shot locked off I guess we will add some fake DOF in post ;(

I will upload the spot when it's done...you guys may like it - kinda funny

Yes, that is 100% percent what I expected, tell us what technique you are now using fro preview. What was the inner loop (what technique was it using)? There are a few things that will cause 50% load on an empty loop like this. Does it look like we can use 1W IGhz VIA Mini-Itx now? Will there be enough power left over on your high end system to do lossless compression?

Very exciting news.

Bye the way, sorry for mentioning you above, have been sick, meant to say "(we had better be nice to Rai and Ronald now" (German Pope ;).

I would like to see some lossless compression, I am not sure yet if that will work..I think a color display would be a better choice of CPU time then compression though...

1ghz Via? I don't think so as we need the pci-x slot for the data from the camera is very very high...unless you know of a via board with pci-x

The Altasens can run up to 150Mpix/sec. Every other frame can be dropped at the camera so you don't clobber the bus with excess frames. Since it is dual tap - two pixels per clock - the camera link clock rate is only 75MHz, well below the CL2 max.

The other thing to watch is the data rate on the bus. During readout, you will be moving 150Mpix * 2 bytes/pix on the PCI-64/66 bus. Then you will have some quiet time for disk accesses.
Steve

Update:
internally we are recording and saving well within the the 24FPS limits. We
should be recording with no problem. There is something going on with the
framebuffer that prevents us from doing so. I will prepare some standartized
tests and try to find out why. It seems now that the limiting factor is with
the PCI bus, but I do not know where. Once I have done that series of tests
I will be able to find out more.

as promised:

www.dv3productions.com/bob.wmv

still has a very TEMP VO on the end and the LOGO work has not been done on it yet....

anyone have some feedback ?

That looks really great Obin! I do some local ad work as well and I must say that it blows away my stuff. You really got alot out of it for being shot on DV, great work man!

here is my latest project http://www.thefacebite.com

Thanks Rob! I kinda like this spot - we had a good crew hair/makeup/wardrobe/gaffer/pa's/I Directed and DP'ed...

DV is so BAD in post ;)

Actually, I would not be surprised if PCI-X and PCI-E turned up this year on their server platform line, but what I was thinking of is the next generation of GigE capture. Actually Steve even offered to implement Lossless if we gave him an FPGA design for it, and Sumix is also "supposed" to be doing it, that's 200MB/s packed data transfer. Still we are very flexible here, 2% preview (and x% capture) on your system should fit on the VIA, and would be a mean little machine. Compression utilising customised (to MB model) GPU code (the original intention for GPU), I can only hope, but even if we have to resort to a 720p version, it still would be a mean little camera.

Thanks

Wayne.

Ohh yes, a question, in the States do you have DVD players that can play foreign DVD's from different regions. I have some locally filmed region 4 DVD's I want to send.

New Altasens senor
 

I have just heard on the grape vine that there will be a new 1/3rd inch Altasens sensor released latter this month. If true, this would be great for an low end cheap option.

Steve, is there any news that you can tell us?

And here is a side thread on the next ProHD EX standard:
http://www.dvinfo.net/conf/showthread.php?t=43386

Yes,
They are doing a 720p sensor to prove out the 3570 architecture. I'm not sure of the value of this for us compared to the Micron used in the SI-3300 that Obin has. You can do about 60fps @720pwith the Micron, it is low cost, low noise and can do 1080p @24fps. OK, you will get true 12 bits instead of 10. We will certainly be doing the 3570.

Higher speeds...?
 

Hi guys,

I am new in this thread and just tried to read fastly over it, but at some point started skipping to the end.

I want to know what do you think of a camera that does smoothly 720/10..250p? Or more?

Would that be cool to anybody around?

Axel

ok weirdness crops up again...we have a stable software now..thing is some strange things are going on inside with the epix card..it holds things up when we try and record...CPU overhead is VERY low now 1-5% ...not sure what it is yet..still working on it...

We have stable 21fps recording now! this is GOOD! with a low CPU overhead and a smooth DISPLAY!/....in the coming days we will be going for the full 24fps record..should not be an issue!

Great news Obin! Can't wait to see your results.

I was not thinking against Micron 1080p chip, but against the cheap 720 capable cameras.

Hi Wayne,

well, cheap enough always depends on your budgets :)

I am currently working on an approach that would be significantly cheaper than many other products. And it could be more capable. It might do things in low frame rates yet unseen on any camera (I am currently doing tests), and it might give you variable speeds in the mentioned range. The second issue is it eats memory like hell.

If I want cheap HD I can run and order an JVC HD100, a really nice gear.

My approach is to sort out if there is interest for such kind of gear, and budget wise what it should cost at max to have any interest.

I am sure it'll be significantly more than 10,000 US$, thats all I can say at now.

Its not aiming at the home user market.

Regards,
Axel

I would also search through all the threads for the other high speed cameras mentioned (I think there was another relatively cheap one).

The obvious thing with these slow motion images, is little variation between images, this leads to great custom compression possibilities through shifting/transforms in the image. That might be a good way to save on disk storage (might be a good idea for the cineform people). What might be able to process such a large amount of information, CELL, other things mentioned in the technical thread, a $299 XBOX2.

While we're here, here's an idea:

Toshiba 3D screen (much better than stereo vision):

http://www.physorg.com/news3773.html

There might be an opportunity to make a camera that can film this sort of thing when these screens go mass market. This thing needs capture that takes lots of images simultaneously (or high speed).

Thanks


Obin, forgot to mention:
VIA has released full source of it's drivers for the Linux community to use. Such a thing should allow significant speed improvements (particularly GPU programming) on supported motherboards.

Something I am sure you are already planning:

Don't worry about getting the software perfect before going to market. As long as it is working correctly you can make money and upgrade latter.

Thanks

See you latter.

I, hopefully, God willing, will pop in every now and again. I am needing to rationalise my time into more constructive areas at the moment. I joined to save myself some money on a good camera and to help in the effort to improve the situation for others. Well it's taken a lot and I can't keep up the effort or even afford one of these cameras without a loan. So I got to clean up, get well, and do something to make money.

it's been a tough week sofar..lots of ups and downs...bottom line is we have things working but the onboard SATA controller is getting in the way of things when we save data..I have placed an overnight order for a really nice SATA pci controller card with 64MB of ram on-board..we think this will fix the issues...

I guess I will know in a few hours..:)

Maybe you could speak with Juan about his Reel Stream Andromeda and get some ideas or pointers.

http://www.reel-stream.com/andromeda

well the $200 "fast" Promise SATA RAID card is about 1/2 the speed of the onboard SATA controller..this makes for a total waste of money spent on the card...would anyone know why this thing is SLOWER then the onboard? I have looked at the IRQ, it's not shared I have looked at all the hardware raid settings, they are all good...it just does not make sense...help anyone?

Obin, will your camera record 4:4:4? To an onboard HDD or to a external PC?

Check the chip set architecture - the on-board controller might be right off the south bridge chip where the Promise is on the PCI bus.

Hi Obin,

I have no idea how many disks you are going to use. We had many different types of SATA controllers tested for editing systems, and the most outstanding to us was the Broadcom RAIDcore series with 4 & 8 drive connections. They do up to 350 MB/s writing with 8 disk in RAID5.

I guess the 4 port version is comparable in price as the Promise you tried.

Axel

Steve do you know of a CL card that has onboard disk controllers?

To have a fast RAID card it needs to be PCI 64 or PCI X.
Also it needs to be placed on a PCI 64 or X slot.

Hi Obin

About Promise SATA RAID slowness - Same case here - the motherboard is "Asus p4p800 E Deluxe" which have 2 SATA RAID controllers
onboard - the first one is intel chipset ICH5R (2 connectors) and the other one is the "famous" Promise SATA RAID (2 connectors too).
All The 4 HDDs are identical Maxtors 200Gb each (I've tested them one by one).

These attached to ICH5R are up to 112-118 MB/s Linear Read/Write
while these on Promise controller are not higher than 50-55 Mb/s !!! LOL

Note- every one single disk of these Maxtors is able to transfer 63-65 Mb/s at his begining !!!

For tests I've used Aida32 (there is a HDD benchmark plugin) and beside the Linear Read/Write tests,there is a Buffered Read test,
which is actually a chip to chip transfer (between HDDs chache memory and SATA controller).
For drives attached to ICH5R the Buffered Read speed was ~120 Mb/s
and for these on Promise ... ta-da-dammm ... can you believe? ... Pure 2000 Mb/s !!!
... I've never liked Promise controllers ... now I hate them!

And these tests (except for Buffered Read) was confirmed by the Blackmagic DeckLink Disk Speed Test.

One more thing- I did some PCI buss overclock and the Buffered Read maximum for ICH5R was 144Mb/s ... at ~42 MHz PCI.
I wonder - is'nt it supposed that speed to be 150 Mb/s for each SATA channel and 300 Mb/s for both at 33MHz PCI ?
Is'nt the Asus(the motherboard maker) the guilty one here ...or these declared SATA speeds are just an another marketing trick ?

P.S. Long time ago :) ...in one of my posts I've say something like "...don't overclock PCI buss if you use SATA HDDs or you'll lose your data...".
indeed I was wrong - in fact at certain PCI frequency (43-44 MHz in my case) the SATA drive is just losing connection to the controller and the computer freezes.
All you needed in that case is to swich off or push the reset button and everything will be OK then.
Need to mention - the overclock must be performed from Windows ( me use SETFSB :) ) -
otherwise if you overclock from BIOS -the SATA drives will fail initialization at boot.

Regards

One thing to consider when choosing a SATA RAID board is whether it does the RAID calculation in hardware or not. This may not be a big deal when doing simple stripe-sets, but when doing RAID 5 with actual redundancy high datarates will actually place a big burden (I've seen 30% CPU load per 100MB/s on Opteron systems) on the CPU.

With Obin's current CPU load issues you wouldn't want to add CPU load for the RAID calculations.

Bar3nd

Obin,
There is one out there that I know of - I/O Industries. They have a superb frame grabber with an on-board SCSI RAID controller. Unfortunately, they are priced out of your range - $3.5K for the FG, $2.5K for the SW. We were able to record 100MPix/sec, 12 bit on a PCI-32 machine without any problems so for applications where you need the highest rates without any bus headaches, it is ideal. For cost sensitive applications, it doesn't fly.
Steve

Hi Steve,

is the I/O controller you mention the one with the 5 times dual 2 GBit FC host ports and CameraLink interface?

Its said to write 850 MByte/s.

Or is it something different you are talking about?

Anyway, is the goal to have Obin's camera on long cable or to have recording "build it". The last one will stay a problem with uncompressed HD for a while :(

Axel

IO Industries - DVR Express:
http://ioindustries.com/express.htm
We have bundled this FG with our cameras in the past. Very nice hardware.

Hi Steve,

yep, thats the one I had in mind too. I didn't know its from I/O industries, anyway, it seems to be the fastest direct to disk solution currently available.

Do you know if there will be PCIexpress x8 or x16 frame grabbers?

The best I found so far do only x4, which is limiting around 500 MByte/s, which is to slow for some projects, which should be enough for this project.

PCIexpress has the big advantage that you can use a typical low-cost SATA RAID controller for writing the data to disk, rather than expensive 5 times FC dual 2 GBit (=10 GBit total) bandwidth. The RAIDs controllers & enclosures alone are expensive enough to buy an HDCAM...



Axel

Axel,
Leutron was talking about PCIe at the Stuttgart Vision show. Matrox is always on the cutting edge of capture speed. Others are planning it. Keep in mind that the camera link buffers are at 85MHz now - that is 85MHz * 2*12 bits per camera link channel. Above that, you run more parallel pathways. Since 500MB/sec is about the full bandwidth of the PCI-64/66 bus, how much more data do you want to move? OK, 3 chip......

As is clamed the 500MB/sec is the full bandwidth of the PCIExpress x 1,
so PCIe x 4 should be 2GByte/s ...no?

Hi Marin, Hi Steve,

I am in contact with Leutron and they are not yet ready with the PCIe version (and most important there are no WinXP64 drivers ready yet, which I would need too).

I too have in mind that PCIe 4x should be faster than 500 MByte/s, but thats what the Leutron person told me. Anyway, for uncompressed standart HD formats something around 250 MBytes/s for RGB 12 bit data should be enough, what do you think? Viper style...

I am more on faster frame speeds for this high bandwidth thingy.

Axel

Hi,

I've just looked at their website (sorry for german here):

Mit PicPort-Express-CL ultraschnelle Datenübertragungen von Zeilen- und Matrixkameras mit bis zu 500 MByte/sec. in den Hauptspeicher des PC´s realisieren.

Camera Link Framegrabber für den PCI Express x1- und x4-Bus
Datenrate x1-Bus: 250 MByte/sec.
x4-Bus: 500 Mbyte/sec.


So thats 250 MByte/s for x1 and 500 MByte/s for x4 type cameras. I do think that a x16 graphics card can do 4000 MByte/s (bi-directional), though that would mean 1x is 16x/16 = 4000 MByte/s / 16 = 250 MByte/s. In that way the equation makes sense to me, but I may be totally wrong.

Axel

Axel,
I don't know what your level of system and software expertise is but please understand that you are talking about moving a tremendous amount of data. System architecture and RAID design should be very well thought out and benchmarked at these rates. It will be very easy to go through many hardware purchases trying to balance a system design at 500MB/sec. Continuous. Important word to add.