Inter-sensory Integration Inter-sensory integration combines information across sensory systems, or across sub-modalities of sensory systems. I study the integration of vision and touch. The two senses obtain information using different physical processes and thus have different strengths and weaknesses. Does the brain take advantage of these differences when information from vision and touch is available at the same time? In a visual-haptic virtual environment we measure how precisely human observers perform simple perceptual tasks with each sense alone, and with both senses together. Then we compare how the single-sense estimates are combined by humans and by a statistically optimal ideal observer. In the tasks that we tested so far we find that human observers approach statistical optimality, so that their combined percepts are more precise than is possible with either sense alone. |
 |
|
Observer's
thumb and index finger are inserted into the thimbles of two force-feedback
devices (PHANToMs) which
are located under
the mirror. PHOTO: Ken Huie, 2003.
|
To achieve that, the nervous system relies more on the sense that provides more precise information, but the less precise senses are taken into account too. The weights assigned to each sense are close to those prescribed by the principle of maximal likelihood. Some of our results are summarized here:
- Gepshtein, S. & Banks, M. S. (2003). Viewing geometry determines how vision and touch combine in size perception. Current Biology 13 (6) p. 483-488. [PDF] [Supplementary Materials]
- Levitan, C. A., Gepshtein, S. & Banks, M. S. (under review). Visual and haptic perception of curved surfaces. [Preliminary results of this study are available in this poster.]