[fluxus] Stereo Fluxus
Alex Norman
alex at neisis.net
Tue May 22 10:33:09 PDT 2007
I guess I should be doing this?
(set-camera
(mmul
(mtranslate (vector half_sep 0 0))
(get-camera-transform)
))
looks good now,
-Alex
On 0, Alex Norman <alex at neisis.net> wrote:
> So I've got stereo partially working in the standalone version of fluxus,
> attached is a diff to the current cvs version... note that I'm setting up a
> stereo window with glut, this will fail to open a window at all if you don't
> support stereo [via your graphics card]. I should add a command line option to
> try to allow for stereo rendering...
>
> it still needs some work but I figured I'd show my progress
>
> one thing is that this:
> (set-camera (mmul (get-camera-transform)
> (mtranslate (vector -1 0 0)))) ; move to the left
>
> translates in the x relative to the center of the "world", not to the screen.
> I'm no master of matrix operations so if someone would send me the operation to
> do the correct translation I'd appreciate it. (in the current mode if you move
> around the world the translation might end up being up and down relative to the
> screen...)
>
> Also, I figure there should be some (set-eye-separation val) and
> (get-eye-separation) functions, I don't see any reason why that needs to be
> implemented anywhere other than scheme land, but I'm not exactly sure how/where
> to implement it. (right now the eye separation is "hard coded")
>
> btw, I tried to get this working in mred, the manual claims that you can create
> a stereo canvas, but I cannot seem to get it working... I've also attached my
> attempt at that.
>
> -Alex
>
>
> On 0, Alex Norman <alex at neisis.net> wrote:
> > On 0, Dave Griffiths <dave at pawfal.org> wrote:
> > >
> > > > Hi,
> > > > I want to modify fluxus to allow for active stereo video [via crystal
> > > > glasses], I've already done this for pd's GEM and it was quite easy,
> > > > though they already had some other stereo modes. Basically, I need to
> > > > render the scene one time for each eye, selecting a different buffer to
> > > > write to for each and offsetting the camera position slightly for each
> > > > eye. I intend to write it in such a way that one could select this
> > > > functionality or not, so that it might actually be put into the main
> > > > distro eventually as others might find it useful (once this stereo mode
> > > > is in place it probably wouldn't be hard to implement other stereo modes).
> > > >
> > > > Anyways, if anyone could suggest where to implement this I'd appreciate
> > > > it (i figure in the renderer, but where in the renderer :)
> > >
> > > Do you want to do this inside the fluxus scratchpad (traditional style) or
> > > with mred? It should be possible to do this with mred now, but I haven't
> > > had a chance to figure out much of mred's gl canvas yet - but you should
> > > be able to get it to change the draw buffer and rerender the scene.
> >
> > I guess I'd like to check out both, I was thinking I'd do it inside the scratch
> > pad but I figure eventually it would be nice to have both options.
> >
> > I'll check into the scratchpad.ss stuff as well as the mred gl canvas to see if
> > it supports stereo windows.
> >
> > thanks,
> > -Alex
> >
> > >
> > > To get it to work in the scratchpad, it'll take some C++ work - I think we
> > > need a low level render command which sets the current buffer to draw to
> > > from scheme (via glDrawBuffer) and enumerate all the options.
> > >
> > > But however it's set, the place to make it work is from the frame
> > > callback, which currently looks like this (and lives inside
> > > scratchpad.ss):
> > >
> > > (define (fluxus-frame-callback)
> > > (set-camera (get-camera-transform))
> > > (framedump-update)
> > > (begin-scene)
> > > (if (not (null? user-callback))
> > > (user-callback))
> > > (end-scene)
> > > (tick-physics)
> > > (update-audio))
> > >
> > > We should be able to change it to something like:
> > >
> > > (define (fluxus-frame-callback)
> > >
> > > ; draw the left buffer
> > > (draw-buffer 'back-left) ; or equivelent mred func call
> > > (set-camera (mmul (get-camera-transform)
> > > (mtranslate (vector -1 0 0)) ; move to the left
> > > (begin-scene)
> > > (if (not (null? user-callback))
> > > (user-callback))
> > > (end-scene)
> > >
> > > ; draw the right buffer
> > > (draw-buffer 'back-right) ; or equivelent mred func call
> > > (set-camera (mmul (get-camera-transform)
> > > (mtranslate (vector 1 0 0)) ; move to the right
> > > (begin-scene)
> > > (if (not (null? user-callback))
> > > (user-callback)) ; need to call the user callback twice, as immediate
> > > ; mode objects will be lost the second time otherwise
> > > (end-scene)
> > >
> > > ; update everything that should only be updated once per frame
> > > (tick-physics)
> > > (update-audio))
> > >
> > >
> ? stereo-5-21-07
> ? stereo-5-22-07
> ? modules/fluxus-audio/src/AudioCollector.os
> ? modules/fluxus-audio/src/FluxusAudio.os
> ? modules/fluxus-audio/src/JackClient.os
> ? modules/fluxus-engine/src/Engine.os
> ? modules/fluxus-engine/src/FluxusEngine.os
> ? modules/fluxus-engine/src/GlobalStateFunctions.os
> ? modules/fluxus-engine/src/LightFunctions.os
> ? modules/fluxus-engine/src/LocalStateFunctions.os
> ? modules/fluxus-engine/src/MathsFunctions.os
> ? modules/fluxus-engine/src/PDataFunctions.os
> ? modules/fluxus-engine/src/PhysicsFunctions.os
> ? modules/fluxus-engine/src/PrimitiveFunctions.os
> ? modules/fluxus-engine/src/SchemeHelper.os
> ? modules/fluxus-engine/src/TurtleBuilder.os
> ? modules/fluxus-engine/src/TurtleFunctions.os
> ? modules/fluxus-engine/src/UtilFunctions.os
> ? modules/fluxus-osc/src/FluxusOSC.os
> ? modules/fluxus-osc/src/OSCCore.os
> ? modules/fluxus-osc/src/OSCServer.os
> ? modules/scheme/scratchpad.ss-old
> Index: examples/12-particles2.scm
> ===================================================================
> RCS file: /sources/fluxus/fluxus/examples/12-particles2.scm,v
> retrieving revision 1.5
> diff -r1.5 12-particles2.scm
> 26c26,27
> < (texture (load-texture "textures/smoke.png"))
> ---
> > ;(texture (load-texture "textures/smoke.png"))
> > (texture (load-texture "/usr/local/src/fluxus-cvs/examples/textures/smoke.png"))
> Index: libfluxus/src/Renderer.cpp
> ===================================================================
> RCS file: /sources/fluxus/fluxus/libfluxus/src/Renderer.cpp,v
> retrieving revision 1.29
> diff -r1.29 Renderer.cpp
> 66a67
> > m_StereoMode(noStereo),
> 695a697,740
> >
> > void Renderer::DrawBuffer(const string &Modename){
> > if(Modename == "back")
> > glDrawBuffer(GL_BACK);
> > else if(Modename == "back-right")
> > glDrawBuffer(GL_BACK_RIGHT);
> > else if(Modename == "back-left")
> > glDrawBuffer(GL_BACK_LEFT);
> > else if(Modename == "front")
> > glDrawBuffer(GL_FRONT);
> > else if(Modename == "front-right")
> > glDrawBuffer(GL_FRONT_RIGHT);
> > else if(Modename == "front-left")
> > glDrawBuffer(GL_FRONT_LEFT);
> > else if(Modename == "right")
> > glDrawBuffer(GL_RIGHT);
> > else if(Modename == "left")
> > glDrawBuffer(GL_LEFT);
> > else if(Modename == "front-and-back")
> > glDrawBuffer(GL_FRONT_AND_BACK);
> > else if(Modename == "none")
> > glDrawBuffer(GL_NONE);
> > else
> > cerr<<Modename<<" is not a valid draw buffer"<<endl;
> > }
> >
> > bool Renderer::SetStereoMode(stereo_mode_t mode){
> > GLboolean stereoWindowTest;
> > switch(mode){
> > case noStereo: m_StereoMode = noStereo;
> > return true;
> > case crystalEyes:
> > //test for a stereo window
> > glGetBooleanv (GL_STEREO, &stereoWindowTest);
> > if(stereoWindowTest){
> > m_StereoMode = crystalEyes;
> > return true;
> > } else {
> > m_StereoMode = noStereo;
> > return false;
> > }
> > };
> > return false;
> > }
> Index: libfluxus/src/Renderer.h
> ===================================================================
> RCS file: /sources/fluxus/fluxus/libfluxus/src/Renderer.h,v
> retrieving revision 1.20
> diff -r1.20 Renderer.h
> 90a91
> > void DrawBuffer(const string &Modename);
> 135a137
> >
> 143a146,148
> > enum stereo_mode_t {noStereo, crystalEyes};
> > bool SetStereoMode(stereo_mode_t mode);
> > stereo_mode_t GetStereoMode(){ return m_StereoMode;}
> 173a179
> > stereo_mode_t m_StereoMode;
> Index: modules/fluxus-engine/src/GlobalStateFunctions.cpp
> ===================================================================
> RCS file: /sources/fluxus/fluxus/modules/fluxus-engine/src/GlobalStateFunctions.cpp,v
> retrieving revision 1.8
> diff -r1.8 GlobalStateFunctions.cpp
> 593a594,662
> > // StartFunctionDoc-en
> > // draw-buffer buffer_name
> > // Returns: void
> > // Description:
> > // Select which buffer to draw in
> > // for stereo mode you'd do 'back-right and 'back-left
> > // Example:
> > // (draw-buffer 'back)
> > // EndFunctionDoc
> >
> > Scheme_Object *draw_buffer(int argc, Scheme_Object **argv)
> > {
> > DECL_ARGV();
> > ArgCheck("draw-buffer", "s", argc, argv);
> > Engine::Get()->Renderer()->DrawBuffer(StringFromScheme(argv[0]));
> > MZ_GC_UNREG();
> > return scheme_void;
> > }
> >
> > // StartFunctionDoc-en
> > // set-stereo-mode mode
> > // Returns: bool
> > // Description:
> > // select which stereo mode to use
> > // Example:
> > // (set-stereo-mode "crystal_eyes")
> > // EndFunctionDoc
> >
> > Scheme_Object *set_stereo_mode(int argc, Scheme_Object **argv)
> > {
> > bool success;
> > std::string val;
> > DECL_ARGV();
> > ArgCheck("set-stereo-mode", "s", argc, argv);
> > val = StringFromScheme(argv[0]);
> > if(val == "crystal-eyes")
> > success = Engine::Get()->Renderer()->SetStereoMode(Renderer::crystalEyes);
> > else if(val == "no-stereo")
> > success = Engine::Get()->Renderer()->SetStereoMode(Renderer::noStereo);
> > else {
> > Engine::Get()->Renderer()->SetStereoMode(Renderer::noStereo);
> > success = false;
> > }
> > MZ_GC_UNREG();
> >
> > if(success)
> > return scheme_true;
> > else
> > return scheme_false;
> > }
> >
> > Scheme_Object *get_stereo_mode(int argc, Scheme_Object **argv)
> > {
> > Renderer::stereo_mode_t mode;
> > DECL_ARGV();
> > mode = Engine::Get()->Renderer()->GetStereoMode();
> > MZ_GC_UNREG();
> >
> > switch(mode){
> > case Renderer::noStereo:
> > return scheme_intern_symbol("no-stereo");
> > case Renderer::crystalEyes:
> > return scheme_intern_symbol("crystal-eyes");
> > default:
> > return scheme_intern_symbol("no_stereo");
> > }
> > }
> >
> >
> 626a696,699
> >
> > scheme_add_global("draw-buffer", scheme_make_prim_w_arity(draw_buffer, "draw-buffer", 1, 1), env);
> > scheme_add_global("set-stereo-mode", scheme_make_prim_w_arity(set_stereo_mode, "set-stereo-mode", 1, 1), env);
> > scheme_add_global("get-stereo-mode", scheme_make_prim_w_arity(get_stereo_mode, "get-stereo-mode", 0, 0), env);
> Index: modules/scheme/scratchpad.ss
> ===================================================================
> RCS file: /sources/fluxus/fluxus/modules/scheme/scratchpad.ss,v
> retrieving revision 1.3
> diff -r1.3 scratchpad.ss
> 221d220
> < (set-camera (get-camera-transform))
> 223,226c222,252
> < (begin-scene)
> < (if (not (null? user-callback))
> < (user-callback))
> < (end-scene)
> ---
> >
> > (if (eq? (get-stereo-mode) 'crystal-eyes)
> > (let ((half_sep (/ 0.5 2)))
> > (draw-buffer "back-left")
> > (set-camera (mmul (get-camera-transform)
> > (mtranslate (vector (- half_sep) 0 0))))
> > (begin-scene)
> > (if (not (null? user-callback))
> > (user-callback))
> > (end-scene)
> >
> > (draw-buffer "back-right")
> > (set-camera (mmul (get-camera-transform)
> > (mtranslate (vector half_sep 0 0))))
> > (begin-scene)
> > (if (not (null? user-callback))
> > (user-callback))
> > (end-scene)
> > (draw-buffer "back")
> > )
> > (begin
> > (draw-buffer "back")
> > (set-camera (get-camera-transform))
> > (begin-scene)
> > (if (not (null? user-callback))
> > (user-callback))
> > (end-scene)
> > )
> > )
> >
> >
> Index: src/main.cpp
> ===================================================================
> RCS file: /sources/fluxus/fluxus/src/main.cpp,v
> retrieving revision 1.50
> diff -r1.50 main.cpp
> 149c149
> < glutInitDisplayMode(GLUT_DOUBLE|GLUT_RGBA|GLUT_DEPTH|GLUT_STENCIL);
> ---
> > glutInitDisplayMode(GLUT_DOUBLE|GLUT_RGBA|GLUT_DEPTH|GLUT_STENCIL|GLUT_STEREO);
> (require
> (lib "gl.ss" "sgl")
> (prefix gl- (lib "sgl.ss" "sgl")))
>
> (load-extension "/usr/local/lib/plt/collects/fluxus-0.12/extensions/fluxus-engine.so")
> (require fluxus-engine)
> ;(fluxus-init)
> (clear-engine)
>
> (define ob '())
>
> (define (setup)
> (clear-colour (vector 0 0 0))
>
> (let ((l (make-light "point" "free")))
> (light-position l (vector 10 50 10)))
>
> ;(hint-unlit)
> (shinyness 80)
> (specular (vector 1 1 1))
> (line-width 4)
> ;(hint-wire)
> (texture (load-texture "/usr/local/src/fluxus-cvs/examples/textures/refmap.png"))
>
> (push)
> (hint-unlit)
> (rotate (vector 180 90 0))
> (scale (vector -10 -10 -10))
> (build-sphere 6 6)
> (pop)
>
> (colour (vector 0.5 0.5 1))
> (scale (vector 3 3 3))
> (set! ob (build-nurbs-sphere 20 20))
> (grab ob)
> ; more reference geometry for the deformation
> (pdata-copy "p" "pref")
> (ungrab))
>
> ; some sinewave deformation with time
> (define (deform n)
> (let ((v (vector (* 1 (sin (+ (flxtime) (* (vector-ref (pdata-get "pref" n) 1) 5.4)))) 0 0)))
> (set! v (vmul v (* (sin (flxtime)) 0.5)))
> (pdata-set "p" n (vadd v (pdata-get "pref" n))))
> (if (< n 0)
> 0
> (deform (- n 1))))
>
>
> ; take the eye and normal vectors and return the texture coordinates of the
> ; reflection vector, calculated by converting them into a spherical lookup
> ; this is a bad quality, but fast - and looks like it might be buggy
> (define reflect
> (lambda (eye normal)
> (let ((refl (vmul (vsub eye normal) (vdot eye normal)))) ; reflection vec
> (vector (+ (/ (vector-ref refl 0) 2) 0.5) ; s coord
> (+ (/ (vector-ref refl 1) 2) 0.5) 0)))) ; t coord
>
> ; this is the interesting part, the facing ratio is the dot product of the direction we are looking
> ; at the vertex from, and the normal of the vertex - where all vectors are normalised. the complex bit
> ; is getting the incident direction, from the camera space transform (see below) and the vertex position
> ; in worldspace.
> (define (toon n camerapos obpos)
> (let ((v (vadd obpos (pdata-get "p" n)))) ; find the vertex in worldspace
> (let ((i (vnormalise (vsub v camerapos)))) ; incident direction (normalised)
> (pdata-set "t" n (reflect i (pdata-get "n" n))))) ; set s to the facing ratio (i dot n)
> (if (< n 0)
> 0
> (toon (- n 1) camerapos obpos)))
>
> (define (render)
> (grab ob)
> (deform (pdata-size))
> (recalc-normals 1)
> (toon (pdata-size)
> ; transforming a vector gets that vector "into" the space of the transform, so 0,0,0 in camera
> ; space is the camera position...
> (vtransform (vector 0 0 0) (get-camera)) ; gets the eye position
>
> ; and 0,0,0 in object space is the object position
> (vtransform (vector 0 0 0) (get-transform))) ; gets the object position
> (ungrab))
>
>
>
> (define fluxus-canvas%
> (class* canvas% ()
> (inherit with-gl-context swap-gl-buffers)
>
> (define/override (on-paint)
> (with-gl-context
> (lambda ()
>
> ; draw the left buffer
> (glDrawBuffer GL_BACK_LEFT)
> (glClearColor 0.0 0.0 0.0 0.0)
> (gl-clear 'color-buffer-bit 'depth-buffer-bit)
>
> ;(set-camera (mmul (get-camera-transform)
> ;(set-camera (mmul (vtransform (vector 0 0 0) (get-camera))
> ; (mtranslate (vector -1 0 0)))) ; move to the left
> (begin-scene)
> (if (null? ob) (setup))
> (render)
> (end-scene)
>
> ; draw the right buffer
> (glDrawBuffer GL_BACK_RIGHT)
> (glClearColor 0.0 0.0 0.0 0.0)
> (gl-clear 'color-buffer-bit 'depth-buffer-bit)
> ;(set-camera (mmul (get-camera-transform)
> ; (mtranslate (vector 1 0 0)))) ; move to the right
> (begin-scene)
> (if (null? ob) (setup))
> (render)
> (end-scene) ; and here, and have to be inside a with-gl-context
>
> (swap-gl-buffers)
> (super on-paint))))
>
> (define/override (on-size width height)
> (with-gl-context
> (lambda () 0)))
>
> (define stereo-gl-config (new gl-config%))
> (send stereo-gl-config set-stereo #t)
> ;(send stereo-gl-config get-stereo)
> (super-instantiate () (style '(gl)) (gl-config stereo-gl-config))))
> ;(super-instantiate () (style '(gl)))))
>
> (define frame (instantiate frame% ("fluxus in mred")))
> (define fluxus-canvas (instantiate fluxus-canvas% (frame) (min-width 640) (min-height 400)))
> (send frame show #t)
>
More information about the Fluxus
mailing list