[Fluxus] [RFC] Fluxus as an A/V batch synth

[Stefano Sabatini]

On date Sunday 2011-06-12 21:13:04 +0200, gabor papp wrote:
> hi Stefano,
> 
> >I'm interested in integrating fluxus with an external application
> >reading/encoding/processing A/V data.
> i was connecting external applications the other way around using
> shared memory. it can be tricky this way, since you have to download
> the data from the gpu to send it to the server, which can be slow.
> if you choose this way, you could try writing an ffgl plugin that
> passes through the pixel primitive content, and sends it to the
> server in shared memory.
> 
> on osx it is possible to use syphon (http://syphon.v002.info/) to
> share frames.
>
> on the other hand, i would use an other machine with a capture card
> if the budget permits. it's simpler and faster, which is quite
> important when dealing with real-time video.
> 
> best,
> gabor

Hi Gabor, I essentially have two problems.

I want to implement a non trivial video synth in FFmpeg for
libavfilter. I could use directly the openGL/other lib features but I
wanted something more flexible (=> scriptable), so what better than
the fluxus beautiful engine and language? 

This was my idea: provide the script filename to the source, and let
it generate the video and send it to the filterchain (no need of
display rendering at this stage, as the objective is batch
processing).

Secondarily, I'd like to connect it with external applications, to
read and pass processed data. With audio there are already existing
solutions (e.g. jackd), with video it looks less trivial (there was a
jack video project but I don't know how far it went). A possible
solution would be to create a virtual output device, and make the
other processes grab from it (e.g. v4l2vd, but still not very
portable).

syphon is very interesting but a) I want a portable solution and b) I
have no mac ;-(.  An alternative path is using streaming, but looks
overkill for local comunication in the same machine (for which using
shared memory looks like a better approach).