OpenGL 3.2 and More

Check out this SlideShare Presentation:

OpenGL 3.2 and More

View more presentations from Mark Kilgard.

Proposal for PROPOSAL

01, Cross-Modality
Sept 21

While I was reading an article on Codex from MOD about research on hyperthermia and hypothermia at QinetiQ, this idea popped up: how about we provide opposite visual signal, giving subjects opposite suggestion about what they are experiencing? With synthetic environment, we can easily generate illusive scene implying that the temperature is not as high as experienced. How much visual suggestion can alter body's feeling of temperature? Not sure. But cross-modal projections do exist.

To this point, I'm thinking more about cross-modal interaction. The facilities in VR lab equipped us for researching on cross-modal interaction. For example, visual/audio interaction (we'll need some audio devices), visual/haptics interaction and also I'm thinking of motion perception on treadmill that involves multisensory procedures as well. With the help of VR Technologies, all of these research can be much more convenient than using traditional methods only.

Back to the thought about hyperthermia and hypothermia, if my assumption serves well, we can handily make synthetic environment deploy-able (to field) by implementing it on mobile computing device. Mobile computing device not only makes Virtual Environment deploy-able to field, actually, any lab (I'm thinking of climatic chambers at QinetiQ, indeed, I mean any lab) without large-scale VR Facilities, can adopt mobile VR now. It does not cost a fortune.

Embedded mplayer

Still, I don't know whether this is useful, but you can try it for fun--I mean it was fun for me. Here it comes:

The sample I post here was created by c# as a WindowsForm, but as I aware there's nothing wrong if you prefer c++.

I'm not sure whether I should rumble about how to create a winform project from Visual Studio, just skip it for now.

Create a winform project, leave everything as default. As showing from the top picture, you drag a button to the form for "stop" and draw another button for "start". I really don't care where you put your buttons or how many you prefer, what they should look like, etc..

Note: at the postion of video window, I put a lable there as pre-opened window, we'll use that window to play media.

Now we come to the job itself, I mean embed mplayer to a window.

Open the Form1.Designer.cs, add a private member:
private System.Diagnostics.Process mplayerP;

Double click the "start" button, it will lead you to the click function:
private void button1_Click(object sender, EventArgs e)
{
int hWnd = mplayerwin.Handle.ToInt32();
string margs = string.Format( @"-wid {0} -vo gl2_stereo yourmedia.wmv", hWnd );
mplayerP = System.Diagnostics.Process.Start("mplayer.exe", margs);
}

Double click the "stop" button:
private void button2_Click(object sender, EventArgs e)
{
mplayerP.Kill();
}

DONE! have fun.

Note: the picture on top was playing video from 3dtv.at, a stereoscopic demo clip.

Three Steps Build a Haptic Model

I suggest you start from simple model, mine is a big H. You can do a sphere, cube as your first try. To keep the model as simple as you can will give you a clear field levels in X3D file. It will be helpful to understand where the haptic field goes from the beginning.

Step 1: Build a normal 3D model

You can use any 3D modeling application you're familiar with. I do not have 3D Max on my computer, so I created my model with blender. It's free and fabulous. Oh, don't forget assign material for your model, a default material will do.

Step 2: Export the model as X3D file

This is too easy to blog anyting on it, just export the file you created.

Step 3: Edit X3D file

Here comes something new. Open the X3D file you just exported with a text editor. Find the field <Appearance> and just after <Appearance> we add a new field <SmoothSurface/>. OK that's all, now you can use H3DLoad to check your model with default SmoothSurface.
Currently, without any programming work we have three haptic surface nodes to choose from. They are:

• SmoothSurfce - a surface without friction
• FrictionalSurface - a surface with friction
• MagneticSurface - makes the shape magnetic

Note: H3DLoad comes with h3dapi package, you can download it from here
Pre requirement: a haptic stylus of course

PS: some H3DLoad options:

• -f for fullscreen
• -m for mirror
• -s for with spacemouse
  

A Simple OpenGL Sample as Firefox plugin

Introduction

It does OpenGL rendering as Firefox plugin. Don't know whether it's useful. I did it for fun!

Background

I was trying to render x3d file with OpenGL as a Firefox plugin. I know there are many such x3d file viewers, but mine intends to handle haptic field...well, this sample is just a by-product.

Tinkering the code

You will first need to download Gecko plugin API and Firefox source code.

Download Gecko SDK extract to \yourroot\gecko-sdk\

Download Firefox source code from here, extract to \yourroot\mazilla. Then we start from a basic plugin sample, come to the folder: yourroot\mozilla\modules\plugin\tools\sdk\samples\basic\windows

Here's a little trick, we need to run unix2dos on npbasic.dsp and npbasic.dsw under this folder. Then we open and cover the project by Visual C++ Express, will be successed now. From project properties change the Additional include directories to yourroot\gecko-sdk\include;yourroot\mozilla\modules\plugin\tools\sdk\samples\include. Try to build now, if successed, we continue...

Open plugin.cpp, after

#include "plugin.h"

add

#include <gl/gl.h>

then after the line

static WNDPROC lpOldProc = NULL; add two functions EnableOpenGL and DisableOpenGL

void EnableOpenGL(HWND hWnd, HDC * hDC, HGLRC * hRC);
void DisableOpenGL(HWND hWnd, HDC hDC, HGLRC hRC);

and in

NPBool nsPluginInstance::init(NPWindow* aWindow)

{

.................

//we add EnableOpenGL and SetTimmer

 EnableOpenGL( mhWnd, &hDC, &hRC );
SetTimer(mhWnd, 0,  1,  (TIMERPROC) NULL); // no timer callback

............

}

then in

void nsPluginInstance::shut()

{

// subclass it back

//We add DisableOpenGL

    DisableOpenGL( mhWnd, hDC, hRC );

....

}

then in PluginWinProc we need to handle WM_TIMMER message like the following

   case WM_TIMER:
   theta += 1.0f;
   PostMessage( hWnd, WM_PAINT, NULL, NULL );
   break;

 

At last we do OpenGL rendering in case WM_PAINT, remove the original code under this case, and replace them with ours as the following:

    glClearColor( 0.0f, 0.0f, 0.0f, 0.0f );
 glClear( GL_COLOR_BUFFER_BIT );
 glPushMatrix();
 glRotatef( theta, 0.0f, 0.0f, 1.0f );
 glBegin( GL_TRIANGLES );
 glColor3f( 1.0f, 0.0f, 0.0f );
 glVertex2f( 0.0f, 1.0f );
 glColor3f( 0.0f, 1.0f, 0.0f );
 glVertex2f( 0.87f, -0.5f );
 glColor3f( 0.0f, 0.0f, 1.0f );
 glVertex2f( -0.87f, -0.5f );
 glEnd();
 glPopMatrix();
 SwapBuffers( hDC );

it's almost done, but don't forget to define EnableOpenGL and DisableOpenGL

void EnableOpenGL(HWND hWnd, HDC * hDC, HGLRC * hRC)
{

    PIXELFORMATDESCRIPTOR pfd;
 int format; // get the device context (DC)
    *hDC = GetDC( hWnd ); // set the pixel format for the DC

    ZeroMemory( &pfd, sizeof( pfd ) );
 pfd.nSize = sizeof( pfd );
 pfd.nVersion = 1;
 pfd.dwFlags = PFD_DRAW_TO_WINDOW | PFD_SUPPORT_OPENGL | PFD_DOUBLEBUFFER;
 pfd.iPixelType = PFD_TYPE_RGBA;
 pfd.cColorBits = 24;
 pfd.cDepthBits = 16;
 pfd.iLayerType = PFD_MAIN_PLANE;
 format = ChoosePixelFormat( *hDC, &pfd );
 SetPixelFormat( *hDC, format, &pfd ); // create and enable the render context (RC)

    *hRC = wglCreateContext( *hDC );
 wglMakeCurrent( *hDC, *hRC );
}

// Disable OpenGL  
void DisableOpenGL(HWND hWnd, HDC hDC, HGLRC hRC)
{
    wglMakeCurrent( NULL, NULL );
 wglDeleteContext( hRC );
 ReleaseDC( hWnd, hDC );
}  
 

Done!!!

Build the Project

Nothing to say, just build it.

Test the plugin

Copy npbasic.dll to C:\Program Files\Mozilla Firefox\plugins\ or yourfolder\Mozilla Firefox\plugins\
From the folder yourroot\mozilla\modules\plugin\tools\sdk\samples\basic\ you will find test.html, double click, you would see a rotating triangle, corlorful...

Another way to test the plugin. If we open a file of the type a plugin registered then the plugin will be triggered. Open basic.rc with a text editor, find the string "FileExtents" and change the value to whatever you want. I changed it to hx3d. Next, create an empty file with the extension .hx3d, I created testbasic.hx3d under my folder. In firefox url box type c:\r.wang\testbasic.hx3d, you would also see the plugin triangle

Now, em....enjoy.
.
.
.
------

OpenRT—Realtime RenderMan Ray-Tracing OpenGL

OpenRT is a realtime ray tracing rendering engine and API. It is designed as powerful as RenderMan by offering all features of ray tracing. And as similar as OpenGL, while as possible. It is RenderMan-like for shader writer and OpenGL-like for application programmers. OpenRT in contrast of OpenGL, supports no immediate mode. This means that every geometry you want to display has to be contained in an OpenRT object. This object has to be instantiated and can then be rendered. This is because ray tracing uses a global approach, not as raster dealing triangles independently.

A non-commercial version of OpenRT can be download from here. But it supports no cluster. And so a dual-core Itanium2 might be a not-so-not-decent processor to run it as a single node. IMHO, OpenRT is designed to run over cluster other than super powerful single node, at least talk against today's computer. I tried the commercial version on my so humble cluster of old old pentium 4s, yet what makes me love it most is that the speedup is so so linear to the number of nodes due to the properties of ray tracing algorithm.

Imagine, realtime RenderMan! Leading digital effects houses and computer graphics specialists use RenderMan and every movie nominated for a Visual Effects Oscar in the last 10 years relied somewhat on RenderMan.

The commercial version is around £40,000 for 16 nodes Itanium2 (IA64). So you see, OpenRT is by no means of Open Source project, only but puns OpenGL.

Ray Tracing vs. Raster Graphics

The rendering technique—ray tracing has reached the stage where it is feasible that it will take over from raster graphics in the near future for interactive simulation/gaming and other application domains. Ray tracing is the act of tracing the trajectory of a ray from one point to another to determine if anything is hit and the distance to the nearest hit point. In fact, ray-tracing techniques can achieve the exact same results as raster-based techniques (including all the approximations and tricks that raster solutions typically require); however, it does not work the other way round.

The primary differentiating factor between raster- and ray tracing approaches is that a ray tracing approach enables one to solve a global problem, while a raster-based approach seeks to achieve similar results by solving a local problem.

Raster graphics attempts to render an image efficiently by making certain convenient assumptions. In particular, it treats triangles as if each triangle is entirely independent of every other triangle. There is a concept of the current state and the current triangle. This is the Graphics Processing Unit's (GPU) view of the entire world; it has no idea if or what comes next. In general, due to the way that raster systems work, they process every pixel of every submitted triangle to determine the final image that needs to be displayed. So raster graphics performance scales strongly with the number of triangles and pixels that have to be processed for a given image, so cost scales strongly linearly with viewport size and overall scene complexity.

Because ray tracing takes a global approach, the natural interface to it is different that that used with raster graphics. Recall that raster graphics use an “immediate mode” interface, and is only aware of a current state and a current primitive at any point in time. Ray tracing, on the other hand, needs a “retained mode” interface where random access to the whole scene is required, and when a visible triangle is determined, a specific shader needs to be invoked on demand. However, the really exciting news is how the ray tracing workload scales. It is strongly effected by the number of rays shot in a scene and weakly effected by the complexity of the scene! Ray tracing scales linearly with the number of rays shot and only logarithmically with the complexity of the scene. For a fixed resolution image, the cost of raster graphics doubles (roughly) as the complexity of the scene doubles; for ray tracing, you would have to increase the viewed scene complexity by 10 times to double the cost. Here we emphasize viewed scene for the ray tracing is not so much overall scene complexity dependent, but dependent upon the visible complexity of the scene. Another amazing fact of ray tracing is that its performance scales linearly with the number of CPUs ( with today's raster oriented graphics accelerators ). And measured linear performance scales up to 128 CPUs (3.2GHz Pentium 4).

5DT Data Glove Successfully Connected to Linux Cluster CAVE Demo

The 5DT Data Glove 16 (right hand) now successfully connected to our Linux cluster city demo, a four-screen CAVE demo. As simple as using pre-defined gesture, index finger, for example to indicate turn right, you can program the gesture as complicated as you want while using raw data instead of pre-defined gestures.

The 5DT Data Glove 16 measures finger flexure as well as the abduction between the fingers. Finger flexure is measured at two places (1^st joint/knuckle, 2^nd joint) on each finger. The glove consists of a lycra glove with embedded fibre optic sensors. The 5DT Data Glove 16 connects to a 9-pin RS232 serial port (DB9 connector) via the interface cable.

The 5DT Data Glove Driver provides access to the 5DT range of data gloves at an intermediate level. The Linux version is provided in form of a C/C++ header file (fglove.h), and a dynamic library file (libfglove.so). The API is free for both Windows and Linux.

There are 16 pre-defined gestures, i.e., 0 for fist and 15 for flat hand, etc.