Summary

The interactive artwork was created using the haptic sense. The system uses a 2D range-finding sensor (LiDAR) to recognize the position where the experiencer touches the wall, which causes actions to occur on the projected content.

Concept

The concept is to "touch the water.” If the glass of an aquarium tank did not exist, what would it feel like to touch a section of water? Based on this question, I attempted to recreate extraordinary experiences and sensations.

System Configuration

Two computers, A and B, were used in the controller unit. A LAN cable connects both computers and communicates data via OSC (open sound control) communication. The range sensor is connected to computer A. It receives the 2D point cloud data from the sensor and locates the obstacle coordinate on the wall. When the sensor detects an object, it sends the object coordinates on the wall surface to Computer B. Computer B generates video content for projection. When computer B receives object coordinates, it generates a video effect at the specific location. In addition, a speaker is connected to Computer B to play background music and sound effects. Furthermore, when command values are sent via serial communication to a microcontroller, the commands are sent to the gloves worn by the user via a wireless communication module, which can control the effects of the glove. In the glove unit, a small microcontroller controls sound, lighting, and vibration effects by the received control signal.

Projected Content

The work is drawn in openFrameworks, an open-source C++ toolkit for creative coding, and a projector projects the app’s execution screen. To allow participants to enjoy the work, the light from the water’s surface, fish, and bubbles are generated as animation. Since these are randomly generated, they can entertain audiences repeatedly. C++ classes were used to render the bubbles. The computer’s memory is released when a randomly generated bubble (an instance of the Bubble class) at the bottom of the screen moves to the top of the screen. This allows multiple bubbles to be rendered repeatedly.

Glove

A glove-type device that synchronizes with video images was also developed. A vibration motor, LEDs, and a speaker are installed on the glove, and it communicates with a laptop that processes the projection of the movie via a wireless module. Wireless communication was used for data communication between the laptop and the globe. That is because the aquarium is projected on a large wall surface, and the experiencer’s range of action will be wider not to use a wired system. When the range sensor connected to the laptop detects the position of the experiencer's hand, a command is sent from the laptop to the glove, and the glove initiates the necessary action.

Experience

• Experience using range sensor data in the C++ language.

• Utilization of C++ class concepts.

• Experience in synchronizing projected content and hardware.