We can all remember the crisply beveled edges of our cheery-yellow No. 2 pencil, the cool, smooth feel of a chalk-powdered blackboard, the gritty red bricks of the schoolhouse walls. Surely that all wasn’t just an illusion?

No, of course not.

But—as it turns out—it kind of is.

The sense of touch (and indeed, all of our senses) is more or less illusory because no sensation stands alone. Stretch out your hands and snap your fingers. This, of course, feels very real. But you’re seeing your fingers, hearing your fingers, and feeling your fingers—and all of these sensations fully correspond.

Now, what if they didn’t?

Virtual Reality (VR) is a great tool for revealing this strange and otherworldly foundation of our everyday sensory perceptions. Sneaky (but, of course, highly ethical) experimentalists such as ourselves can render a completely computer-generated world. If we hand you a pair of controllers that can vibrate on command, we can play tricks.

We can even use this simple apparatus to produce a sensation of touch that “feels like” it originates in the completely empty space between your outstretched hands—an experiment we discuss in our paper in the current issue of Science Robotics.

This VR equivalent of a carnival trick is known as funneling. Each controller provides the user with synchronous vibrotactile stimuli of different amplitudes. In the virtual environment, we show the participant a wooden dowel that they grip with each hand. The dowel is only imaginary, of course—the hands are not physically linked, and the dowel does not exist. We then show a small white marble that seems to knock on the dowel at different locations as we vary the vibratory sensations.

This is where it gets weird. The apparent spatial location of the touch sensation will persist, even if the marble is invisible to the eyes of the beholder. Your brain fills in these gaps and inconsistencies—the vibratory sensations felt by your hands, the (at times imaginary) little white marble knocking on the dowel in the physically empty space in-between. A particular spatial location that happens to be completely outside of your own body.

This is a neat perceptual illusion, but why does it matter?

Well, in virtual environments (or robotic tele-operation tasks), producing higher-fidelity haptic (that is, tactile) sensations of this sort come with the (oft-unstated) assumption that such “improvements” will always yield more a realistic and immersive experience. Using this funneling illusion when the marble was imaginary we can demonstrate an “uncanny valley” effect for haptic sensations, in which the distorted sensations of touch produced in our experiments that were almost right.

As a result, they were exactly wrong, and the immersion was broken.

But this uncanny valley can be shifted or eliminated by subtly manipulating the experimental conditions. For example, if the dowel appears to be inside a smoky-looking cylinder, thus partially obscuring the location of the knocking (but invisible) marble, this provides a plausible reason for any discrepancy between the perceived location of the haptic sensation, versus what we see with our eyes—and the simulation feels more realistic.

Likewise, the sensation can be manipulated based on whether we are simply holding the virtual dowel—thus experiencing the knock of the marble as a passive participant—or actively moving the dowel up and down, thus becoming the ‘agent’ of the sensation through our own actions.

Although demonstrated in our curious VR testbed at present, these effects are rooted in human perception. As such, understanding them more deeply helps us build better and more convincing virtual environments.

SOURCE: https://blogs.scientificamerican.com/observations/if-virtual-reality-feels-almost-right-its-exactly-wrong/