Shape Perception of 3D Virtual Objects by Tactile Feedback Systems
by Massimo Bergamasco
The capability to touch and
manipulate virtual objects according to natural patterns of hand movements
is a fundamental requirement for future applications of Virtual Environment
technologies. In the real world, the recognition of peculiar features,
such as geometrical shape, weight, hardness, thermal conditions and surface
texture are performed by exploratory procedures of the hand which are actively
controlled by the user (observer) in order to maintain contact with the
object. The perceptual system devoted to the recognition of such features
is called 'haptics'. The sensory information exploited by the haptics system
for the recognition of real objects are kinesthetic and cutaneous inputs.
While kinesthetic inputs refer to the perception of the spatial configuration
of the hand and fingers, the cutaneous inputs deal with the perception
of the contact conditions between the human hand and the real object (cutaneous
input derives primarily from tactile and proprioceptive sensors).
The availability of an interface system capable of generating adequate
cutaneous stimuli on the user's hand is a primary need when the problem
of the recognition of features of an object is translated from a real to
a virtual environment in which the objects are simulated in the computer.
Such interface systems are called haptic interfaces or haptic displays
and possess functionalities:
- to record the movements of the real hand and fingers in order to replicate
them in the virtual environment
- to generate contact force and tactile stimuli on the user's hand.
The user, by means of a haptic display, is able to perform the correct
pattern of movement as in a real environment, while perceiving artificial
cutaneous stimuli during the contact with the virtual object.
The research activity at the PERCRO laboratory of the Scuola Superiore
S. Anna, Pisa, is primarily devoted to the development of complete virtual
environment systems including haptic interfaces. The haptic display applications
developed at PERCRO refer to experimental activities for the shape and
surface texture recognition of virtual objects. We present here the interface
system employed for experiments on shape and texture recognition together
with the preliminary results.
The haptic display system
The complete VE system for the control of exploratory procedures of
virtual objects consists of a set of software modules running on a Silicon
Graphics VGX440 workstation and of a haptic display worn by the human operator.
The software modules utilized are:
- modeling and real-time visualization module for the graphic representation
of the virtual scenario (virtual hand, virtual object, environment)
- collision detection module to determine contact areas between the virtual
hand and the virtual object
- force generation module calculating the contact force for each contact
area
- tracking module for the acquisition of the hand movement data deriving
from a tracking sensor and the kinesthetic sensors of the haptic display.
The last version of the glove-like advanced interface.
The haptic display consists of a sensorized glove with tactile effectors
integrated at the level of the fingertips. The glove possesses 20 kinesthetic
sensors located in correspondence with each phalanx: movements of flexion-extension
and abduction-adduction can be recorded and reproduced in the VE for each
finger. The tactile effectors have the functionality of generating both
thermal and indentation stimuli on the fingertips. Each effector consists
of a 4x4 array of microsolenoids controlled by a microcontroller to reproduce
micro and macroscopic geometrical features of the virtual object; a thermal
effector is integrated with the previous array and is capable of generating
the adequate heat flow in correspondence of the contact area with the fingertips.
Experimental activity
Experiments have been carried out by users wearing the sensorized glove
with the tactile effectors. The virtual scenario representing the virtual
hand and one of the virtual objects (from simple geometrical polyhedra
to complex architectural artifacts) is shown on the screen of the graphic
workstation. The user controls the movements of the hand and fingers through
a tracking sensor and the kinesthetic sensors of the glove, respectively.
The user can start the exploration procedure of the surface shape of the
virtual object by performing natural movements of the hand/fingers. The
tactile effectors exploit the information coming from the collision detection
module to generate the correct pattern of indentation according to the
relative position of the virtual fingers on the surface of the virtual
object, the relative velocity of exploration and the depth of interpenetration
between the virtual object's surface and the virtual hand. The user controls
the movements of his/her hand by exploiting the mechanical (indentation
and thermal) stimuli generated by the haptic display.
The user can perform exploratory procedures of virtual objects by successfully
recognizing geometrical features such as holes, edges, vertexes, etc. with
only cutaneous and kinesthetic feedback. The same result can also be obtained
when no direct visual feedback of the operation is provided. We intend
to integrate force feedback capabilities into the haptic display shortly.
The introduction of anthropomorphic haptic interfaces into a Virtual Environment
system must be considered as a fundamental functionality for the control
of operations of direct manipulation, grasping and exploration of virtual
objects in the near future.
The haptic device and its application described in this paper have been
developed under the Basic Research contract No. 6358 SCATIS funded by the
European Union and the CNR project Conoscenze per Immagini. For more information,
see: http://www-percro.sssup.it/
Please contact:
Massimo Bergamasco - PERCRO, Scuola Superiore S. Anna
Tel: +39 50 883289
E-mail: massimo@percro.sssup.it
