[fluxus] Video Texture

[Dave Griffiths]

>
>> Not yet, but a discussion of what people want would be very handy for
>> me.
>> At the moment I cant decide the best way to proceed.
>
> as far as feature requests go...
>
> - frame limit should not be an arbitary power of 2, but would probably be
> memory limited

Well you either have to stretch them or add black borders at some point,
its possible to do it on the fly, but it will be slooow.

The other option (I think) is to upload to an oversized power of 2 texture
and wangle texture coordinates to fit, but communicating the information
to the scheme script to implement that will be a pain.

> - looping without pauses is important, and something that pix found
> troublesome in his experiments with ogre >
> http://libarynth.fo.am/cgi-bin/view/Libarynth/VideoTexture
>
> - the ability to support diff. resolutions, and bitdepths, but not
> necesarily manual rescaling.
>
> - being able to use the same mapping as textures, ie wrapped or tiled.

It should be exactly the same deal as with any other textures I think.
Mind you - I haven't exposed clamping or repeating or mipmap settings for
them yet...

> - smooth sequential replacement of textures [prob. related to looping]

Whats this? Linear interpolation between frames?

> - layered textures? alpha, being able to use the gl blendmodes...

This is an interesting one. I'd like to either find a video format that
supports alpha (+ arbitary channels ideally) or use a second greyscale
video as an alpha map.

> in most ways, i thinks the interface should follow that of img textures.
> the
> 'load' command could use optional args or keywords for things like
> looping,
> framerate, etc+ the manupulation [eg. playback speed, sequencing] requires
> a
> bit more thought, but ideally would be similar, eg.
> (framerate video-texture 25) or just (framerate 25) for a grabbed object.
> .

Yes, this is where it gets more tricky.
Maybe movies should be an extension of pixel primitives - they are
grabbable and can be operated on via pdata. OTOH I don't want to add a
bunch of commands which operate on the grabbed primitive but only of
certain types, that's going to get confusing.

Maybe we need a whole new thing called a pixels object or something, which
works like the pixel primitive except that it can't be rendered, only used
as something to make textures from, whether procedurally, or from images,
or from movies. Then we can do things like:

(define mypixels (build-pixels))

(pixels-grab mypixels)
(pixels-load "mymovie.ogg")
(pixels-ungrab)

...

(pixels-grab mypixels)
(pixels-set-frame! (modulo (frame) (pixels-get-numframes)))
(texture (pixels-make-texture))
(draw-cube)     ; cube is texture mapped with the movie frames
(pixels-ungrab)

You could edit the pixel values of images or the current frame of movies
with pdata commands, and even save them back out to disk.

I like this slightly lower level approach as it means I don't have to have
a million commands for different playback options, and you can do realtime
processing of movies with pdata commands to do different things.

> (of course, there are a few edge cases, such as what happens when changing
> the
> direction of a keyframe based interpolated format (eg. mpeg) but these
> could
> provide some interesting compost.. .. ) ))

I guess if the codec doesn't support random access, then anything other
than incrementing frames will be ignored (or be slow as it will have to
reset the stream every frame).

cheers,

dave