Courtesy Jerome Pasquero/McGill University

DaVinci's paintbrush. By tapping a pattern on a fingertip, this array can create an optical illusion

What You See Is What You Feel

Stare at a waterfall long enough, and nearby stationary objects such as rocks and trees will seem to drift up. The optical illusion is called motion aftereffect, and it may trick more than just your eyes, according to a new study. When subjects watched a stationary stripe on a computer screen after a machine stroked their fingertips, the motion of the stroking created the illusion that the stripe was moving. The discovery demonstrates for the first time a two-way crosstalk between touch and vision, challenging long-held notions of how the brain organizes the senses.

In 2000, neuroscientist Christopher Moore, now at the Massachusetts Institute of Technology in Cambridge, noticed that when a vibrating device buzzed a subject's fingertips, the visual motion detector in the brain fired up. But he immediately dismissed the result. At the time, researchers thought that the brain processes each sense--taste, touch, smell, sight, and hearing--separately and that it only later combines them to interpret the world.

Over the past 5 years, however, further studies have challenged this picture. Experiments with blind subjects, for example, have found that reading Braille by touch can trigger activity in the brain's visual cortex (ScienceNOW, 8 November 2002). Researchers attributed the phenomenon to the brain rewiring itself to compensate for disability. But Moore and graduate student Talia Konkle wondered if the sight-touch link might lurk in everyone and if one sense might influence the other.

So they turned to "DaVinci," a delicate, apple-sized tactile stimulator that taps out patterns with a centimeter-square array of pins (see picture and first video here). The researchers asked eight volunteers to watch as dark stripes fell or rose on a white screen (see first image here). Then DaVinci tapped a stationary stripe on their fingertips. Another eight subjects felt DaVinci sweep its pins upward or downward along their fingertips while they viewed a stationary stripe onscreen.

To the researchers' surprise, the motion aftereffect transferred between the senses. Subjects who watched the moving stripes rise perceived the stationary row of pins to be falling along their finger, and vice versa. Those who had been "swept" by DaVinci also saw the stationary onscreen stripe moving in the direction opposite the sweep, the researchers report online today in Current Biology.

"These illusions happen between senses," says Moore, "and that suggests a much deeper connection between them." The discovery means that, contrary to the classical view, touch can shape what we see.

"This is an important advance with some unexpected results," says Jon Kaas, a neuroscientist at Vanderbilt University in Nashville, Tennessee. "Not long ago, we wouldn't have thought there's any tactile-visual interplay at all."

Moore and Konkle plan to pin down where in the brain the sensory conversation occurs in hopes that might lead to new rehabilitation methods for victims of strokes or other brain injuries that can impair vision or sensation.

Posted in Brain & Behavior