1. technology and computing
2. software

VR system components
VR - software, the missing piece?
n
DISTRIBUTION/
COMMUNICATION
HARDWARE
A VR system (or game) can consist of one or more of the following
items:
n
Visual display
n
Image generator
n
n
n
SIMULATION
n
INTERACTION
n
n
2002-11-26
©Anders Backman, Dept. Computing Science, VR02 - Software
n
n
n
n
n
n
Physical structure
Projectors
Computer
Peripherals
n
n
n
Device drivers
Support tools
Application
n
n
©Anders Backman, Dept. Computing Science, VR02 - Software
3
n
Magnetic tracker 30-144Hz
n
A low report rate must not slow the application down.
Availability
n
What happens if a trackersystem breaks down?
n
Application crash?
Configuration
VR - More than just graphics
n
n
n
Design is critical
Reuse
The use of design patterns can help getting a
common language between implementors.
2002-11-26
©Anders Backman, Dept. Computing Science, VR02 - Software
4
A lot of work to accomplish immersive VR
int main(int argc, char **argv) {
// Init
// Loop forever and do:
// Read devices
VRdevice_read();
// Update objects
VRupdate_objects();
// Render the Scene
VRrender_scene();
}
Software libraries from many
vendors/projects could possible be involved
n
©Anders Backman, Dept. Computing Science, VR02 - Software
Software
It is an integration of technologies
n
Different setups for different applications
2002-11-26
Why software?
n
Changing from one tracker to another may demand a total
rewrite of the application when poorly done.
Different report rates
n
2002-11-26
2
Compatibility
n
Software:
n
©Anders Backman, Dept. Computing Science, VR02 - Software
Some integration issues
Hardware:
n
Drivers
2002-11-26
Integration
n
Heterogenous, TCP, UDP, LAN, Modem,...
Software
n
1
Operating system, single multi CPU, shared/distributed memory...
Network
n
HUMAN ISSUES
SOFTWARE
APPLICATIONS
Gloves etc..
Computing environment
n
REALTIME ASPECTS
Headspeaker, 2-4-8 speakers
Interaction devices
n
GRAPHICS
Camera based, magnetic, ultrasound, ...
Sound system(s)
n
n
One graphics card, Multi-pipe onyx, Cluster of PC:s
Tracking system(s)
n
SOUND
Computer scree, CAVE, Powerwall, ...
n
5
Tools are needed
2002-11-26
©Anders Backman, Dept. Computing Science, VR02 - Software
6
1
VE system architecture
Software
There are VR tools for lots of different areas:
n
graphics
n
Modeling
n
Translating
n
Authoring (events, …)
n
Display
electronics
n
Virtual
Environment
Generation
system
head position
hand
position
(Grapics
software and
workstation)
Head- and
handtracking
electronics
Immersive
Visualization
Facility
n
n
Device support
Rendering
n
Audio
localizer
n
Distribution/Communication
Sound
n
Simulation
n
2002-11-26
©Anders Backman, Dept. Computing Science, VR02 - Software
n
7
2002-11-26
Software - Modeling
n
n
n
n
n
n
n
n
n
n
n
Home-brewers way of life:
n
Great for modelling, easy to use
No real control over polygons.
Easy to write plugins to get full control
No support for hiearcheal fileformat except VRML
n
n
n
n
n
n
n
Best for Rendering
©Anders Backman, Dept. Computing Science, VR02 - Software
n
9
n
n
n
3D modeling tool (3D Studio, LightWave)
Graphics (OpenGL), building a “graphic engine”
Home written device drivers, if any.
2002-11-26
3D Studio/Maya
Special tools for
Special tools for
Special tools for
+ Export plugin.
lighting (generating Lightmaps)
animations, scripting, ...
everything.
©Anders Backman, Dept. Computing Science, VR02 - Software
n
Remember the whole picture
DISTRIBUTION/
COMMUNICATION
HARDWARE
Low-level: API:s
High-Level: GUI, drag-n-drop, Word for VR!!
SIMULATION
For prototyping a higher-level tool is often desirable
Sometime low-level API give you the control you need over details
Different tools are suitable for different purposes (special purpose
tools)
SOUND
INTERACTION
GRAPHICS
n
n
n
n
n
Very complicated to support all of these areas in an effective but still
general way.
The VR area are missing standards, work in progress.
Result ⇒ Most VR authoring tools are still in its infancy, work in progress.
Some nice systems exists though.
Development cost is high
n
n
2002-11-26
10
Software
Tools are in different levels
n
Hard to make a general product that lives up to all that.
Game Developers way of life:
Great at NURBS, although low control over polygons
Software
n
8
There are also tools that tries to match all of the above points.
But:
LightWave
AC3D
Blender
…
2002-11-26
n
©Anders Backman, Dept. Computing Science, VR02 - Software
n
n
Alias
Maya
n
n
Great for real-time modeling
Full control over polygons!
Hiearcheal file format.
Rhino3D
n
n
Handling of spatial sound connected to objects
Simulation code for physical /artificial entities.
Software
3D Studio
n
n
Handling the consistency of a distributed DB.
Multigen
n
n
Communicates with the external VR hardware (trackers…)
Graphics, usually a SceneGraph API on top of OpenGL
n
n
Haptic
system
Converting file-formats
Handling the actions in the application.
n
n
Audio
synthesizer
Creating the 3D models
©Anders Backman, Dept. Computing Science, VR02 - Software
11
2002-11-26
Many platforms, high-end to low end
Lot of hardware issues
©Anders Backman, Dept. Computing Science, VR02 - Software
12
2
Software - File-formats
Software - Converters
There are a lot of 3D file-formats out there
Problems arise when trying to convert between them.
Materials, textures disappear, ...
n
n
n
n
VRML 1 .WRL
n
VRML 2 .WRL
n
n
n
n
n
n
n
n
n
n
First standard, no events and such, only geometry
n
Second standard, contains a lot: interpolation, sound, key framing, ...
No support for named objects
n
n
Defacto standard for Vis-sim business.
Hiearcheal format
Supports links to other .flt files
n
n
Contains no textures or materials
AutoDesk .3DS
n
n
n
AutoDesk .DXF
n
There are a lot of converters out there too:
n
MultiGen’s OpenFlight .FLT
n
n
n
3D studio mesh format.
Geometry (mesh+textures) + animations.
Wavefront .OBJ
WorldToolKit Neutral File Format NFF
Inventor .IV
Every scenegraph API usually have its own native fileformat.
n
2002-11-26
©Anders Backman, Dept. Computing Science, VR02 - Software
13
2002-11-26
Software - Graphics
n
n
n
n
n
n
n
n
n
Industry standard for accessing the rendering hardware in an portable
way.
View models interactively in 3D space
Control color and lightning
manipulate pixels...
Portable
Full control
In most cases to low level to start from scratch developing a whole
immersive VR application
n
n
n
n
n
n
n
n
n
n
n
n
n
n
n
One of the first portable graphic API:s, now also OpenSource
A mature multiplatform API.
©Anders Backman, Dept. Computing Science, VR02 - Software
n
15
2002-11-26
Software - Graphics
Support for 6DOF, Stereo, Collision detection, sound..
SGI/NT/Linux
- Slow
+ Java = Portable, stable?
VRML
See OpenGL except for portability!!
2002-11-26
Aimed towards CAD industry
Tessellation, optimization, Occlusion culling, multiprocessor support (SGI).
NT, SGI (SUN, HP)
Almost shutdown from SGI.
Java3D (www.sun.com)
OpenInventor
n
14
Optimizer/Cosmo3D (www.sgi.com)
Direct3D
n
n
n
OpenGL,
n
©Anders Backman, Dept. Computing Science, VR02 - Software
Software - Graphics
Graphic APIs.
n
PolyTrans from Nugraf
Crossroads (freeware)
ivToflt, wrlToflt, … (SGI only)
Some work, some don’t. (In some cases)
60% of work with a VR project can directly be related to
fileconversion problems.
Make sure you have a workflow ready.
A well developed process of the whole chain of data, from
generating it to rendering/simulating it.
Not an API but still powerful
Portable
Performance?
Outgrowing it self?
X3D
Mostly used as a fileformat. Not as a script language.
©Anders Backman, Dept. Computing Science, VR02 - Software
16
Software - Graphics
§ OpenScenegraph (www.openscenegraph.org)
§ A Core scenegraph
n
n
§ Handles the scene only, no windowing, user interaction, simulation etc...
This is supposed to be handled by additional toolkits and libraries.
Plib (www.plib.org)
OpenSG (www.opensg.org)
n
n
§ Well object orientated, using Designpatterns.
§ Open source, get too peek at the source, AND to modify it.
§ Readers for most common formats, the community adds that.
n
n
German research project
Core scenegraph
Multithreaded core design
Gizmo3D (www.tooltech.com)
n
n
§ OpenFlight, 3DS, AC3D, VRML1, LWO, OBJ, GEO
n
n
§ Portable
n
Swedish company
Mature
Readers for most common filetypes
Strictly OO
Not OpenSource, but free for academic use.
§ Unix, Windows, MacOs X, Solaris, HP, (Playstation2)
§ Impostors
n
Performer (www.sgi.com)
§ First implementation in a scenegraph.
n
§ Occlusion culling
§ -Under heavy development
n
n
n
§ Things changes (interfaces, functionality)
§ Reaching 1.0, wich will mean a more stable core.
n
n
§ - Documentation
n
n
§ Code examples the best way to get into it.
§ Documentation is on the way.
2002-11-26
©Anders Backman, Dept. Computing Science, VR02 - Software
n
n
17
2002-11-26
Optimized for SGI (but now exists for Linux and Windows)
A lot of functionality
Multiprocessing
System stress and load management
Asynchronous database paging
Morphing
Clip Mapping for handling of really large textures.
Fixed-frame-rate capability
DVR (Dynamic Video Resolution) (On SGI Onyx)
Not really easy to work with.
©Anders Backman, Dept. Computing Science, VR02 - Software
18
3
Software - Hardware?
n
Software
Device drivers
n
n
n
n
n
A VR toolkit should contain most of the following issues
If you choose to program from scratch in Performer, OpenGL, ... for
the graphics, you usually have to communicate with the hardware
yourself.
Different trackers have different ways of communication, different
data-structures, …
Makes it hard to write yourself
To use finished products help a lot.
n
Real time graphics
n
Importing models, filters
n
Model libraries
n
Animation
n
Collision detection
n
n
n
n
n
2002-11-26
©Anders Backman, Dept. Computing Science, VR02 - Software
19
n
n
n
Networking
n
n
n
n
n
n
n
n
Support for trackers, gloves, ...
n
n
Exchanging minimal level of data over a network so several users can
cooperate in a VE.
Dead reckogning algorithms
Interest groups
Support for different projection systems
©Anders Backman, Dept. Computing Science, VR02 - Software
21
2002-11-26
VR Juggler
n
n
n
n
n
n
n
n
n
n
n
n
n
22
Commercial
n
IDE/API: Multigen, SmartScene, VEGA Prime
n
VEGA is one of the most competent VR development toolkits available.
Lots of different modules for:
n
n
n
Write on a PC running windows, using mouse for interaction
Later run on a multipipe onyx with trackersystem
n
n
Support for cluster rendering
Sound support
Does not include a scenegraph, depends on either OpenGL, Performer,
OSG, OpenSG
Sounds wonderful, wheres the catch?
n
©Anders Backman, Dept. Computing Science, VR02 - Software
Software
University of Iowa
Open source
Developed to make code once, compile and run anywhere for VR.
Incorporates hardware support with display support.
GUI:s for setting up the environment.
You dont have to decide the interaction hardware while writing the
code. This is configurated later on.
n
CAVElib, Commercial
Maverick Open Source, AIG, UK
DIVE, SICS, Sweden.
Simple Virtual Environment (SVE), Georgia Tech, USA.
MR Toolkit, University of Alberta, Canada
DIVERSE, Open Source (built on Performer)
VR Juggler, Open Source, Iowa State University
Stereo Viewing
n
n
20
Research
Inverse kinematics, rigid-body-dynamics, ...
n
Bounding box, geometry
©Anders Backman, Dept. Computing Science, VR02 - Software
n
Sensors
Interpolators, movie-clips, ...
Listing of VR software
Controlling 3D sound sources and listeners, loading of different sound files.
n
n
n
Simulation
n
2002-11-26
Where actions are programmed without need for compiling, etc. Trigger for
actions.
Audio
n
Primitives: box, polygons, spheres, cones, ...
Software
Scripting language
n
CAD models can be surface patches or NURBS or CSG (Constructive Solid
Geometry)
2002-11-26
Software
n
Level-of-detail, culling, ...
n
Marine simulation - Dynamic ocean simulation
Radar simulation
Special effects - Explosions, smoke, fire, missile trail…
Terrain generation
Sounds a lot like the US Defence to me?
Expensive
Large
Monumental software
Many requirements of tools/versions for getting it compiled.
2002-11-26
©Anders Backman, Dept. Computing Science, VR02 - Software
23
2002-11-26
©Anders Backman, Dept. Computing Science, VR02 - Software
24
4
Software
n
Sense8
n
n
n
n
n
API: WorldToolKit
IDE: WorldUp
World2World
IDE: WorldUp from Sense8
Integrated Development Environment
n
Eon Reality’s EON
Superscape VR
n
Muse Technologies Inc.
n
n
n
n
Gone out of business.
Gone out of business
Dvise
n
n
Software
Not developed anymore.
Application development Interfaces
n
n
n
2002-11-26
Boston Dynamics Inc.
DI guys human simulation
Magma from ReachIn Technologies
©Anders Backman, Dept. Computing Science, VR02 - Software
25
2002-11-26
Software
n
n
n
2002-11-26
n
Used for ergonomic evaluations
Developed at University of Pennsylvania
Has 74 segments, 73 joints, 22 segment spine, 150DOF
Can be customized based on scaling, obeys joint and strength limits
taken from NASA studies, TJ:s dimensions is based on 132
anthropometric studies
©Anders Backman, Dept. Computing Science, VR02 - Software
n
n
27
dVise from Parametric Technology Corp (former Division)
Suitable for CAD prototyping
Video
2002-11-26
Software
n
n
28
Simulation driven or graphic driven?
n
Init graphics()
SetupScene()
SetupDevices()
SetupActions()
while(1) do
readDevices()
callActions()
updateModels()
renderGraphics()
enddo
©Anders Backman, Dept. Computing Science, VR02 - Software
©Anders Backman, Dept. Computing Science, VR02 - Software
Software
A general VR application
2002-11-26
26
Software
IDE: EAI’s Transom Jack
n
©Anders Backman, Dept. Computing Science, VR02 - Software
n
Usually the simulation has been pushed away for the graphics.
This is undergoing a change. For example the games requires more
and more precise simulation underneath. AI, physics, …
n
n
This is getting more and more important.
It always depends on what you are looking for.
n
n
29
2002-11-26
In an Immersive Visualization system where people is trying to find pockets
of oil from measurements of the earth’s soil, maybe an authentic simulation
of the friction is not of primarily interest.
For a vehicle simulator, where the vehicle’s interaction with the terrain is
important, it is obvious that some sort of physical simulation is needed.
©Anders Backman, Dept. Computing Science, VR02 - Software
30
5
Software
n
Software
A graphic driven simulation will prioritize the graphics. The
attributes of the graphic entity is given to the simulation
n
n
Graphic representation
(scene graph)
Simulation representation
A
B
Simulation code for C
If User enter C then
Do Action C
C
D
Whereas a simulation driven simulation is based on the entities in the
simulation.
Rather then building the graphical scene and than add dynamic behaviour,
there is a simulation which contain all the information needed to simulate
and generate the visible parts of the object.
Mass,
inertia,
speed, ...
E
Mass,
inertia,
speed, ...
Simulation code for E
In each timeframe check for
contact with D (ground)
2002-11-26
©Anders Backman, Dept. Computing Science, VR02 - Software
31
2002-11-26
Software
n
Mass,
inertia,
speed, ...
Graphics
©Anders Backman, Dept. Computing Science, VR02 - Software
32
Software
Some software packages exists with behavioral built into it.
n
Generates
n
Vortex
n
Havok
EON - Contains springs and multibody
n
n
A virtual tour of Tutankhamuns tomb.
n
n
A Walk around a town square
Physics SDK
Plugins for 3D Studio
Modelica
n
n
Reality Training™ example, where the user
is practicing odontological techniques on a
virtual patient, utilizing a force-feedback
device.
Physics SDK
A research project in Linköping.
Generic language for simulations.
n
n
Rigid body, fluid, electronics, ...
Transom Jack, lots of inverse kinematics and simulation
around the soft tissues in the body.
Reality Training™ example, where the user is practicing
to dismantle a car brake disc.
2002-11-26
©Anders Backman, Dept. Computing Science, VR02 - Software
33
2002-11-26
©Anders Backman, Dept. Computing Science, VR02 - Software
34
And not to mention - the WEB
n
n
3D chat/model builders
Real-time multi-user interaction in 3D environments
n
n
n
n
n
n
n
n
Active Worlds
Adobe Atmosphere
Blaxxun
Ultimate 3D Chat
TalkWorld
Bang
V-Chat
Deep Matrix
2002-11-26
©Anders Backman, Dept. Computing Science, VR02 - Software
35
6