The vast majority of us is used to interact with 2D objects, such as a computer screen. But how do you deal with a volumetric display, such as an architectural model? In this short article, "Gestures control true 3D display," Technology Research News (TRN) writes that researchers from the University of Toronto have devised a method which involves a multi-finger gestural interaction with the 3D display. The users, who carry 'markers' on their fingers which are tracked by cameras, can pick, manipulate or control objects existing in the 3D environment. As the TRN article was only wetting my appetite, I've done my own research on the subject. And among other facts, I discovered that these computer scientists won the Best Paper Award at the 17th annual ACM symposium on User interface software and technology (UIST 2004). Read more.
Here is what says TRN.
Researchers from the University of Toronto have put together a system that allows for direct gestural interaction with virtual objects contained in a volumetric display.
The researchers' method involves using fingers to gesture in the space around and on the surface of the volumetric display. The user's finger positions and postures are tracked by a set of four cameras.
The interface includes two-dimensional menus projected on the surface of the display and a browser for selecting three-dimensional objects used to construct models. The browser uses a grid that contains three-dimensional images of objects like cubes, spheres and pyramids.
Using finger gestures, users can point at objects, make gestures to trigger commands, and manipulate three-dimensional models projected in the display, including moving, rotating and resizing the models or portions of the models.
It's time to look at pictures, which will explain better the concept.
I mistrust the satisfaction which makes a display of the possession of Infinity; that is called fatuity in philosophic terms.
—Edgar Quinet (1803–1875)
Here you can see a user interacting with a volumetric display. A camera-based motion tracking system looks at the 'markers' attached to his fingers (Credit: University of Toronto)
And here you can see all the pieces of the volumetric display (Credit: University of Toronto)
Finally, here is how you can interact with the 3D volumetric display using your fingers: (a) point posture: index finger points towards the display. (b) flat posture: index finger is parallel to display surface. (c) pinch posture: tips of index finger and thumb brought together. (d) curl posture: tip bent towards base of finger. (e) trigger gesture: thumb presses against index finger (f) scrub gesture: thumb scrubs along index finger in either direction.(Credit: University of Toronto)
This research work, from Tovi Grossman, Daniel Wigdor, and Ravin Balakrishnan, has been presented at the 17th annual ACM symposium on User interface software and technology (UIST 2004), which was held in Santa Fe in October 2004.
Here are two links to the Proceedings of this conference and to the abstract of the researchers' paper,"Multi-finger gestural interaction with 3d volumetric displays."
Volumetric displays provide interesting opportunities and challenges for 3D interaction and visualization, particularly when used in a highly interactive manner. We explore this area through the design and implementation of techniques for interactive direct manipulation of objects with a 3D volumetric display. Motion tracking of the user's fingers provides for direct gestural interaction with the virtual objects, through manipulations on and around the display's hemispheric enclosure. Our techniques leverage the unique features of volumetric displays, including a 360° viewing volume that enables manipulation from any viewpoint around the display, as well as natural and accurate perception of true depth information in the displayed 3D scene. We demonstrate our techniques within a prototype 3D geometric model building application.
Here is a link to the full paper (PDF format, 10 pages, 4.77 MB)
The photographic enthusiast likes to lure us into a darkened room in order to display his slides on a silver screen. Aided by the adaptability of the eye and by the borrowed light from the intense projector bulb, he can achieve those relationships in brightness that will make us dutifully admire the wonderful autumn tints he photographed on his latest trip. As soon as we look at a print of these photographs by day, the light seems to go out of them. It is one of the miracles of art that the same does not happen there. The paintings in our galleries are seen one day in bright sunshine and another day in the dim light of a rainy afternoon, yet they remain the same paintings, ever faithful, ever convincing. To a marvelous extent they carry their own light within. For their truth is not that of a perfect replica, it is the truth of art.
—E. H. Gombrich (b. 1909)
For even more information, you should check the University of Toronto's Dynamic Graphics Project.
Sources: Technology Research News, December 29, 2004/January 5, 2005; and various websites
Related stories can be found in the following categories.
Displays
Future
Human Computer Interface
Vision and Visualization.