[Fluxus] Raster video status

[Dave Griffiths]

Hi Ian,

> "I. E. Smith-Heisters" <public at 0x09.com> writes:
>
>> Hi all,
>>
>> Looking at the fluxus FAQ I see
>> ---
>> Can I use fluxus to mix and blend video clips?
>> Fluxus is a 3D renderer, so no. But plans are afoot to extend it in
>> this direction using video textures.
>> ---
>
> There is a video plugin "availible" to test as of now and is supposed to
> work only with the 0.12 release, you can take a look at it here:
>
> http://www.pawfal.org/Software/fluxus/files/fluxus-video-texture-0.1.tar.gz

It should work on 0.13 too... the video texture extension is meant as a
way of using a video as an object's texture, and using hardware to process
it (blendmodes, texture coordinate warping, GPU shaders etc) - so it could
already be used to blend and mix video with quite a large range of
effects.

However, as you mention OpenCV, I'm guessing you are talking more about
computer vision, and using a live camera feed as an input device...

>> But I think that was there last time I looked 6 months ago. Does
>> anybody know the status of this part? Could I lend a hand? I'm an able
>> C/Lisp coder with raster graphics experience, though I'm not too
>> familiar with Fluxus yet.
>
> Fluxus is coded in Cpp and shouldnt be a real problem to understand it
> for a experienced coder i think as it isnt really big, lots of stuff
> though ;).
>
>> I'd be wanting to do some rather advanced stuff with it, not just
>> fades. Is support for something like OpenCV possible, or is that out
>> of scope for Fluxus?
>
> My wild guess is that Dave doesnt want to do it himself as he does
> similar stuff at work and Fluxus is a mean of abstract out of it(he can
> explain it better), but then i dont know talk to him :), also show us
> some of your previous work if you feel like so, the main goal of fluxus
> is livecoding and its derivatives that might arise, though it is an
> open source project developed by open minded people. We are more devoted
> to 3d as the visual principle but raster stuff is very welcomed i think.

I've been struggling with this issue for a while, but I'll put my neck out
and say it - I don't think fluxus is the place for video processing code.

Mainly, as you say, it's scope. Fluxus is designed in the role of a
prototyping games engine - the Scheme processing granularity is really
per-object and at a push per-vertex but really not per-pixel (which in
this context means offloading the work to the GPU).

Video processing and computer vision are massive fields - I'm having a
hard enough time supporting the set of features fluxus has already :)

Also, as Glauber mentioned, computer vision is my day job, and there are
two issues with that, one is my interest level in doing it in my free
time, and the other is annoying my employer.

Would *would* work really well I think (in terms of computer vision),
would be to use osc to drive fluxus from a dedicated application
responsible for computer vision processing - and if needbe display the
camera feed from a plugin like the video texture extension.

Other apps would benefit from such a project too, as it's really just
another form of input device.

What do you think?

cheers,

dave