3D Finger Gesture Recognition used in the Manipulation of Floating Images

We are collaborating with a team in Japan to produce interaction methods for floating images. Floating images are those projected into free air to display information, with which the user can easily interact, since there are no enclosures or screens. It looks similar to the iconic holographic video system from the movie Star Wars (see photo).

Since the image is projected to free air, however, it is difficult to sense the position of the fingers of the user. Different methods have been proposed and implemented such as real-time image processing, ultrasonic transducers or even short range radar.

In this project we will implement user finger position using LIDARs (light flight of time sensors) to detect the user. A practical device will be built to be used in future experiments of user interaction with floating images.