linuxaudio.org

--0016e6d9a32946c9470479b78e30

Content-Type: text/plain; charset=ISO-8859-1
This seems to be due to the massive amounts of IO needed for realtime video
When I was working with YUV4Linux (basically a DVD stream without encryption

or compression) for a 2 GB file I was getting 2 minutes and 18 seconds at

PAL DV resolution. Raw rgb should be almost double that and rgba more than

double. HD resolutions will double and quadruple the amount of

memory/storage needed.
To get data between applications I was using Fifos. Which work, but are

difficult to set up and tempremental at the best of times.
I don't think it would be that realistic to work with compression between

applications in jack type situation. though colour space compression should

be possible
to give some idea of the memory/disk IO requirements needed.

I did some quick calculations
comparing a pal dv sized stream 720 x 576 @ 25 frames a second

vs 48 000 samples per second floating point audio stream.
rgba 32bit float = 864 48k audio streams.

rgba 8bit int = 216 48k audio streams.

rgb 8 bit int =  162  48k audio streams

YCrCb 422 = 108 48k audio streams

YCrCb 420 = 81 48k audio streams
for 720p

rgb 8bit = 360 48k audio streams

YCrCb 420 = 180  " " "
for 1080p

rgb 8bit = 810 48k audio streams
Are there any other color spaces I should be worried about?
Most compositors use RGBA internally either as 8bit int or floats.

FreiOr is a set of video plugins used by a lot of different video editing

programs it prefers rgba 8888, (bgra8888 is the other option) so that format

must at least be acceptable to a lot of video editors. Effectv is the other

shared video plugin set I am aware of and it uses rgb888.

Cinelerra supports RGBA float, RGBA 8888 & YUVA 8888 and non Alpha varients

of the above.
Perhaps the most sensible Idea would be to concider purhaps RGB(A) 888(8)

and YUV(A) 888(8) as different types of port.
On Wed, Dec 2, 2009 at 11:33 AM, Paul Davis wrote:
> On Tue, Dec 1, 2009 at 8:02 PM, Danni Coy  wrote:

quoted text
> > You would be suprised. RGBA32 is used quite commonly in the compositor

> > world, usually stored as a sequence of images, with even higher

> precisions

> > available. Raw RGB24 is reasonably common with the video professionals I

> > know, when I questioned them about it they said that they could acheive

> more

> > film like colour grading than they could in any of the YUV colour spaces.

> >

> > It's probably not the best format for realtime work given the amount of

> data

> > that needs to be transmitted (or maybe it is), I have previously worked

> with

> > piping raw YUV (yuv4linux) between applications and having something like

> > jack to manage the connections would be a godsend.

>

> my understanding is that one of the many problems that video faces is

> that there truly is no equivalent to "32 bit floating point for

> audio". whereas this format for audio samples is pretty much

> acceptable for just about all purposes, and the ones not satisfied by

> it are corner cases, in video there are many common cases that prefer

> quite different data representations. for an audio analogy, imagine a

> JACK world where some clients wanted FFT-bin data while others wanted

> floating point encoding of PCM.

>

> until or unless someone can step up and authoritatively say "the

> interchange format for video is XXXXXX", its hard to imagine a "jack

> for video" system really working very usefully for a significant

> number of people. those who work entirely in the RGBA32 or YUV spaces

> would be happy with just that format; anyone mixing processing that is

> better suited to different formats is going to take a hell of a

> performance hit.

>

> --p

>

--0016e6d9a32946c9470479b78e30

Content-Type: text/html; charset=ISO-8859-1

Content-Transfer-Encoding: quoted-printable
This seems to be due to the massive amounts of IO needed for realtime video=

 When I was working with YUV4Linux (basically a DVD stream without =

encryption or compression) for a 2 GB file I was getting 2 minutes and 18 s=

econds at PAL DV resolution. Raw rgb should be almost double that and rgba =

more than double. HD resolutions will double and quadruple the amount of me=

mory/storage needed.

To get data between applications I was using Fifos. Which work, but are=

 difficult to set up and tempremental at the best of times.I don&#3=

9;t think it would be that realistic to work with compression between appli=

cations in jack type situation. though colour space compression should be p=

ossible

to give some idea of the memory/disk IO requirements needed.I did s=

ome quick calculations comparing a pal dv sized stream 720 x 576 @ =

25 frames a secondvs 48 000 samples per second floating point audio str=

eam.

rgba 32bit float =3D 864 48k audio streams.rgba 8bit int =3D 216 48=

k audio streams.rgb 8 bit int =3D=A0 162=A0 48k audio streamsYCrCb =

422 =3D 108 48k audio streamsYCrCb 420 =3D 81 48k audio streams=

for 720p

rgb 8bit =3D 360 48k audio streamsYCrCb 420 =3D 180=A0 " " &q=

uot;for 1080prgb 8bit =3D 810 48k audio streamsAre ther=

e any other color spaces I should be worried about?Most compositors=

 use RGBA internally either as 8bit int or floats.

FreiOr is a set of video plugins used by a lot of different video editing p=

rograms it prefers rgba 8888, (bgra8888 is the other option) so that format=

 must at least be acceptable to a lot of video editors. Effectv is the othe=

r shared video plugin set I am aware of and it uses rgb888.

Cinelerra supports RGBA float, RGBA 8888 & YUVA 8888 and non Alpha vari=

ents of the above.Perhaps the most sensible Idea would be to concid=

er purhaps RGB(A) 888(8) and YUV(A) 888(8) as different types of port.

On Wed, Dec 2, 2009 at 11:33 AM, Paul Da=

vis <pau=

l@linuxaudiosystems.com> wrote:

On Tue, Dec 1, 2009 at 8:02 PM, Danni Coy <danni.coy@gmail.com> wrote:

quoted text
> You would be suprised. RGBA32 is used quite commonly in the compositor=

> world, usually stored as a sequence of images, with even higher precis=

ions

quoted text
> available. Raw RGB24 is reasonably common with the video professionals=

 I

quoted text
> know, when I questioned them about it they said that they could acheiv=

e more

quoted text
> film like colour grading than they could in any of the YUV colour spac=

es.

quoted text
>

> It's probably not the best format for realtime work given the amou=

nt of data

quoted text
> that needs to be transmitted (or maybe it is), I have previously worke=

d with

quoted text
> piping raw YUV (yuv4linux) between applications and having something l=

ike

quoted text
> jack to manage the connections would be a godsend.

my understanding is that one of the many problems that video faces is=
that there truly is no equivalent to "32 bit floating point for

audio". whereas this format for audio samples is pretty much

acceptable for just about all purposes, and the ones not satisfied by

it are corner cases, in video there are many common cases that prefer

quite different data representations. for an audio analogy, imagine a

JACK world where some clients wanted FFT-bin data while others wanted

floating point encoding of PCM.
until or unless someone can step up and authoritatively say "the

interchange format for video is XXXXXX", its hard to imagine a "j=

ack

for video" system really working very usefully for a significant

number of people. those who work entirely in the RGBA32 or YUV spaces

would be happy with just that format; anyone mixing processing that is

better suited to different formats is going to take a hell of a

performance hit.
--p
--0016e6d9a32946c9470479b78e30--