[fluxus] Stereo Fluxus
Alex Norman
alex at neisis.net
Tue May 22 10:01:43 PDT 2007
So I've got stereo partially working in the standalone version of fluxus,
attached is a diff to the current cvs version... note that I'm setting up a
stereo window with glut, this will fail to open a window at all if you don't
support stereo [via your graphics card]. I should add a command line option to
try to allow for stereo rendering...
it still needs some work but I figured I'd show my progress
one thing is that this:
(set-camera (mmul (get-camera-transform)
(mtranslate (vector -1 0 0)))) ; move to the left
translates in the x relative to the center of the "world", not to the screen.
I'm no master of matrix operations so if someone would send me the operation to
do the correct translation I'd appreciate it. (in the current mode if you move
around the world the translation might end up being up and down relative to the
screen...)
Also, I figure there should be some (set-eye-separation val) and
(get-eye-separation) functions, I don't see any reason why that needs to be
implemented anywhere other than scheme land, but I'm not exactly sure how/where
to implement it. (right now the eye separation is "hard coded")
btw, I tried to get this working in mred, the manual claims that you can create
a stereo canvas, but I cannot seem to get it working... I've also attached my
attempt at that.
-Alex
On 0, Alex Norman <alex at neisis.net> wrote:
> On 0, Dave Griffiths <dave at pawfal.org> wrote:
> >
> > > Hi,
> > > I want to modify fluxus to allow for active stereo video [via crystal
> > > glasses], I've already done this for pd's GEM and it was quite easy,
> > > though they already had some other stereo modes. Basically, I need to
> > > render the scene one time for each eye, selecting a different buffer to
> > > write to for each and offsetting the camera position slightly for each
> > > eye. I intend to write it in such a way that one could select this
> > > functionality or not, so that it might actually be put into the main
> > > distro eventually as others might find it useful (once this stereo mode
> > > is in place it probably wouldn't be hard to implement other stereo modes).
> > >
> > > Anyways, if anyone could suggest where to implement this I'd appreciate
> > > it (i figure in the renderer, but where in the renderer :)
> >
> > Do you want to do this inside the fluxus scratchpad (traditional style) or
> > with mred? It should be possible to do this with mred now, but I haven't
> > had a chance to figure out much of mred's gl canvas yet - but you should
> > be able to get it to change the draw buffer and rerender the scene.
>
> I guess I'd like to check out both, I was thinking I'd do it inside the scratch
> pad but I figure eventually it would be nice to have both options.
>
> I'll check into the scratchpad.ss stuff as well as the mred gl canvas to see if
> it supports stereo windows.
>
> thanks,
> -Alex
>
> >
> > To get it to work in the scratchpad, it'll take some C++ work - I think we
> > need a low level render command which sets the current buffer to draw to
> > from scheme (via glDrawBuffer) and enumerate all the options.
> >
> > But however it's set, the place to make it work is from the frame
> > callback, which currently looks like this (and lives inside
> > scratchpad.ss):
> >
> > (define (fluxus-frame-callback)
> > (set-camera (get-camera-transform))
> > (framedump-update)
> > (begin-scene)
> > (if (not (null? user-callback))
> > (user-callback))
> > (end-scene)
> > (tick-physics)
> > (update-audio))
> >
> > We should be able to change it to something like:
> >
> > (define (fluxus-frame-callback)
> >
> > ; draw the left buffer
> > (draw-buffer 'back-left) ; or equivelent mred func call
> > (set-camera (mmul (get-camera-transform)
> > (mtranslate (vector -1 0 0)) ; move to the left
> > (begin-scene)
> > (if (not (null? user-callback))
> > (user-callback))
> > (end-scene)
> >
> > ; draw the right buffer
> > (draw-buffer 'back-right) ; or equivelent mred func call
> > (set-camera (mmul (get-camera-transform)
> > (mtranslate (vector 1 0 0)) ; move to the right
> > (begin-scene)
> > (if (not (null? user-callback))
> > (user-callback)) ; need to call the user callback twice, as immediate
> > ; mode objects will be lost the second time otherwise
> > (end-scene)
> >
> > ; update everything that should only be updated once per frame
> > (tick-physics)
> > (update-audio))
> >
> >
-------------- next part --------------
? stereo-5-21-07
? stereo-5-22-07
? modules/fluxus-audio/src/AudioCollector.os
? modules/fluxus-audio/src/FluxusAudio.os
? modules/fluxus-audio/src/JackClient.os
? modules/fluxus-engine/src/Engine.os
? modules/fluxus-engine/src/FluxusEngine.os
? modules/fluxus-engine/src/GlobalStateFunctions.os
? modules/fluxus-engine/src/LightFunctions.os
? modules/fluxus-engine/src/LocalStateFunctions.os
? modules/fluxus-engine/src/MathsFunctions.os
? modules/fluxus-engine/src/PDataFunctions.os
? modules/fluxus-engine/src/PhysicsFunctions.os
? modules/fluxus-engine/src/PrimitiveFunctions.os
? modules/fluxus-engine/src/SchemeHelper.os
? modules/fluxus-engine/src/TurtleBuilder.os
? modules/fluxus-engine/src/TurtleFunctions.os
? modules/fluxus-engine/src/UtilFunctions.os
? modules/fluxus-osc/src/FluxusOSC.os
? modules/fluxus-osc/src/OSCCore.os
? modules/fluxus-osc/src/OSCServer.os
? modules/scheme/scratchpad.ss-old
Index: examples/12-particles2.scm
===================================================================
RCS file: /sources/fluxus/fluxus/examples/12-particles2.scm,v
retrieving revision 1.5
diff -r1.5 12-particles2.scm
26c26,27
< (texture (load-texture "textures/smoke.png"))
---
> ;(texture (load-texture "textures/smoke.png"))
> (texture (load-texture "/usr/local/src/fluxus-cvs/examples/textures/smoke.png"))
Index: libfluxus/src/Renderer.cpp
===================================================================
RCS file: /sources/fluxus/fluxus/libfluxus/src/Renderer.cpp,v
retrieving revision 1.29
diff -r1.29 Renderer.cpp
66a67
> m_StereoMode(noStereo),
695a697,740
>
> void Renderer::DrawBuffer(const string &Modename){
> if(Modename == "back")
> glDrawBuffer(GL_BACK);
> else if(Modename == "back-right")
> glDrawBuffer(GL_BACK_RIGHT);
> else if(Modename == "back-left")
> glDrawBuffer(GL_BACK_LEFT);
> else if(Modename == "front")
> glDrawBuffer(GL_FRONT);
> else if(Modename == "front-right")
> glDrawBuffer(GL_FRONT_RIGHT);
> else if(Modename == "front-left")
> glDrawBuffer(GL_FRONT_LEFT);
> else if(Modename == "right")
> glDrawBuffer(GL_RIGHT);
> else if(Modename == "left")
> glDrawBuffer(GL_LEFT);
> else if(Modename == "front-and-back")
> glDrawBuffer(GL_FRONT_AND_BACK);
> else if(Modename == "none")
> glDrawBuffer(GL_NONE);
> else
> cerr<<Modename<<" is not a valid draw buffer"<<endl;
> }
>
> bool Renderer::SetStereoMode(stereo_mode_t mode){
> GLboolean stereoWindowTest;
> switch(mode){
> case noStereo: m_StereoMode = noStereo;
> return true;
> case crystalEyes:
> //test for a stereo window
> glGetBooleanv (GL_STEREO, &stereoWindowTest);
> if(stereoWindowTest){
> m_StereoMode = crystalEyes;
> return true;
> } else {
> m_StereoMode = noStereo;
> return false;
> }
> };
> return false;
> }
Index: libfluxus/src/Renderer.h
===================================================================
RCS file: /sources/fluxus/fluxus/libfluxus/src/Renderer.h,v
retrieving revision 1.20
diff -r1.20 Renderer.h
90a91
> void DrawBuffer(const string &Modename);
135a137
>
143a146,148
> enum stereo_mode_t {noStereo, crystalEyes};
> bool SetStereoMode(stereo_mode_t mode);
> stereo_mode_t GetStereoMode(){ return m_StereoMode;}
173a179
> stereo_mode_t m_StereoMode;
Index: modules/fluxus-engine/src/GlobalStateFunctions.cpp
===================================================================
RCS file: /sources/fluxus/fluxus/modules/fluxus-engine/src/GlobalStateFunctions.cpp,v
retrieving revision 1.8
diff -r1.8 GlobalStateFunctions.cpp
593a594,662
> // StartFunctionDoc-en
> // draw-buffer buffer_name
> // Returns: void
> // Description:
> // Select which buffer to draw in
> // for stereo mode you'd do 'back-right and 'back-left
> // Example:
> // (draw-buffer 'back)
> // EndFunctionDoc
>
> Scheme_Object *draw_buffer(int argc, Scheme_Object **argv)
> {
> DECL_ARGV();
> ArgCheck("draw-buffer", "s", argc, argv);
> Engine::Get()->Renderer()->DrawBuffer(StringFromScheme(argv[0]));
> MZ_GC_UNREG();
> return scheme_void;
> }
>
> // StartFunctionDoc-en
> // set-stereo-mode mode
> // Returns: bool
> // Description:
> // select which stereo mode to use
> // Example:
> // (set-stereo-mode "crystal_eyes")
> // EndFunctionDoc
>
> Scheme_Object *set_stereo_mode(int argc, Scheme_Object **argv)
> {
> bool success;
> std::string val;
> DECL_ARGV();
> ArgCheck("set-stereo-mode", "s", argc, argv);
> val = StringFromScheme(argv[0]);
> if(val == "crystal-eyes")
> success = Engine::Get()->Renderer()->SetStereoMode(Renderer::crystalEyes);
> else if(val == "no-stereo")
> success = Engine::Get()->Renderer()->SetStereoMode(Renderer::noStereo);
> else {
> Engine::Get()->Renderer()->SetStereoMode(Renderer::noStereo);
> success = false;
> }
> MZ_GC_UNREG();
>
> if(success)
> return scheme_true;
> else
> return scheme_false;
> }
>
> Scheme_Object *get_stereo_mode(int argc, Scheme_Object **argv)
> {
> Renderer::stereo_mode_t mode;
> DECL_ARGV();
> mode = Engine::Get()->Renderer()->GetStereoMode();
> MZ_GC_UNREG();
>
> switch(mode){
> case Renderer::noStereo:
> return scheme_intern_symbol("no-stereo");
> case Renderer::crystalEyes:
> return scheme_intern_symbol("crystal-eyes");
> default:
> return scheme_intern_symbol("no_stereo");
> }
> }
>
>
626a696,699
>
> scheme_add_global("draw-buffer", scheme_make_prim_w_arity(draw_buffer, "draw-buffer", 1, 1), env);
> scheme_add_global("set-stereo-mode", scheme_make_prim_w_arity(set_stereo_mode, "set-stereo-mode", 1, 1), env);
> scheme_add_global("get-stereo-mode", scheme_make_prim_w_arity(get_stereo_mode, "get-stereo-mode", 0, 0), env);
Index: modules/scheme/scratchpad.ss
===================================================================
RCS file: /sources/fluxus/fluxus/modules/scheme/scratchpad.ss,v
retrieving revision 1.3
diff -r1.3 scratchpad.ss
221d220
< (set-camera (get-camera-transform))
223,226c222,252
< (begin-scene)
< (if (not (null? user-callback))
< (user-callback))
< (end-scene)
---
>
> (if (eq? (get-stereo-mode) 'crystal-eyes)
> (let ((half_sep (/ 0.5 2)))
> (draw-buffer "back-left")
> (set-camera (mmul (get-camera-transform)
> (mtranslate (vector (- half_sep) 0 0))))
> (begin-scene)
> (if (not (null? user-callback))
> (user-callback))
> (end-scene)
>
> (draw-buffer "back-right")
> (set-camera (mmul (get-camera-transform)
> (mtranslate (vector half_sep 0 0))))
> (begin-scene)
> (if (not (null? user-callback))
> (user-callback))
> (end-scene)
> (draw-buffer "back")
> )
> (begin
> (draw-buffer "back")
> (set-camera (get-camera-transform))
> (begin-scene)
> (if (not (null? user-callback))
> (user-callback))
> (end-scene)
> )
> )
>
>
Index: src/main.cpp
===================================================================
RCS file: /sources/fluxus/fluxus/src/main.cpp,v
retrieving revision 1.50
diff -r1.50 main.cpp
149c149
< glutInitDisplayMode(GLUT_DOUBLE|GLUT_RGBA|GLUT_DEPTH|GLUT_STENCIL);
---
> glutInitDisplayMode(GLUT_DOUBLE|GLUT_RGBA|GLUT_DEPTH|GLUT_STENCIL|GLUT_STEREO);
-------------- next part --------------
(require
(lib "gl.ss" "sgl")
(prefix gl- (lib "sgl.ss" "sgl")))
(load-extension "/usr/local/lib/plt/collects/fluxus-0.12/extensions/fluxus-engine.so")
(require fluxus-engine)
;(fluxus-init)
(clear-engine)
(define ob '())
(define (setup)
(clear-colour (vector 0 0 0))
(let ((l (make-light "point" "free")))
(light-position l (vector 10 50 10)))
;(hint-unlit)
(shinyness 80)
(specular (vector 1 1 1))
(line-width 4)
;(hint-wire)
(texture (load-texture "/usr/local/src/fluxus-cvs/examples/textures/refmap.png"))
(push)
(hint-unlit)
(rotate (vector 180 90 0))
(scale (vector -10 -10 -10))
(build-sphere 6 6)
(pop)
(colour (vector 0.5 0.5 1))
(scale (vector 3 3 3))
(set! ob (build-nurbs-sphere 20 20))
(grab ob)
; more reference geometry for the deformation
(pdata-copy "p" "pref")
(ungrab))
; some sinewave deformation with time
(define (deform n)
(let ((v (vector (* 1 (sin (+ (flxtime) (* (vector-ref (pdata-get "pref" n) 1) 5.4)))) 0 0)))
(set! v (vmul v (* (sin (flxtime)) 0.5)))
(pdata-set "p" n (vadd v (pdata-get "pref" n))))
(if (< n 0)
0
(deform (- n 1))))
; take the eye and normal vectors and return the texture coordinates of the
; reflection vector, calculated by converting them into a spherical lookup
; this is a bad quality, but fast - and looks like it might be buggy
(define reflect
(lambda (eye normal)
(let ((refl (vmul (vsub eye normal) (vdot eye normal)))) ; reflection vec
(vector (+ (/ (vector-ref refl 0) 2) 0.5) ; s coord
(+ (/ (vector-ref refl 1) 2) 0.5) 0)))) ; t coord
; this is the interesting part, the facing ratio is the dot product of the direction we are looking
; at the vertex from, and the normal of the vertex - where all vectors are normalised. the complex bit
; is getting the incident direction, from the camera space transform (see below) and the vertex position
; in worldspace.
(define (toon n camerapos obpos)
(let ((v (vadd obpos (pdata-get "p" n)))) ; find the vertex in worldspace
(let ((i (vnormalise (vsub v camerapos)))) ; incident direction (normalised)
(pdata-set "t" n (reflect i (pdata-get "n" n))))) ; set s to the facing ratio (i dot n)
(if (< n 0)
0
(toon (- n 1) camerapos obpos)))
(define (render)
(grab ob)
(deform (pdata-size))
(recalc-normals 1)
(toon (pdata-size)
; transforming a vector gets that vector "into" the space of the transform, so 0,0,0 in camera
; space is the camera position...
(vtransform (vector 0 0 0) (get-camera)) ; gets the eye position
; and 0,0,0 in object space is the object position
(vtransform (vector 0 0 0) (get-transform))) ; gets the object position
(ungrab))
(define fluxus-canvas%
(class* canvas% ()
(inherit with-gl-context swap-gl-buffers)
(define/override (on-paint)
(with-gl-context
(lambda ()
; draw the left buffer
(glDrawBuffer GL_BACK_LEFT)
(glClearColor 0.0 0.0 0.0 0.0)
(gl-clear 'color-buffer-bit 'depth-buffer-bit)
;(set-camera (mmul (get-camera-transform)
;(set-camera (mmul (vtransform (vector 0 0 0) (get-camera))
; (mtranslate (vector -1 0 0)))) ; move to the left
(begin-scene)
(if (null? ob) (setup))
(render)
(end-scene)
; draw the right buffer
(glDrawBuffer GL_BACK_RIGHT)
(glClearColor 0.0 0.0 0.0 0.0)
(gl-clear 'color-buffer-bit 'depth-buffer-bit)
;(set-camera (mmul (get-camera-transform)
; (mtranslate (vector 1 0 0)))) ; move to the right
(begin-scene)
(if (null? ob) (setup))
(render)
(end-scene) ; and here, and have to be inside a with-gl-context
(swap-gl-buffers)
(super on-paint))))
(define/override (on-size width height)
(with-gl-context
(lambda () 0)))
(define stereo-gl-config (new gl-config%))
(send stereo-gl-config set-stereo #t)
;(send stereo-gl-config get-stereo)
(super-instantiate () (style '(gl)) (gl-config stereo-gl-config))))
;(super-instantiate () (style '(gl)))))
(define frame (instantiate frame% ("fluxus in mred")))
(define fluxus-canvas (instantiate fluxus-canvas% (frame) (min-width 640) (min-height 400)))
(send frame show #t)
More information about the Fluxus
mailing list