Overview

Magical Painter is an interactive installation in which pixie dust appears in response to the participant’s hand movements, creating the sensation of controlling magic.

Objectives

• Create an interactive experience that allows participants to feel as though they are using magic.

• Explore creative coding using Processing.

• Learn how to use a Kinect motion sensor.

Development Process

Idea

Based on the concept of “using magic,” I decided to create an experience in which participants could control pixie dust.

Ideally, I wanted the interaction to take place within a three-dimensional space. However, at the time, I did not yet have the technical knowledge or a practical method to realize this idea. I therefore developed the experience using visuals projected onto a screen.

A Kinect sensor was installed in front of the screen to track the participant’s hand position. Pixie dust then appeared on the screen in response to their hand movements.

System Configuration

The system consists of four main components: a Kinect sensor, a PC, a projector, and speakers.

The PC processes depth data captured by the Kinect to determine the position of the participant’s hands. An application developed in Processing uses this data to generate and display the pixie-dust graphics.

The application is displayed through a projector and projected onto the screen. Sound effects are played through speakers installed on both sides of the screen.

Custom-Built Speakers

For this project, I built the speakers myself as an exercise in electronics and to gain a better understanding of how audio amplifiers work.

Although the sound quality was limited compared with commercially available speakers, building a functional audio system from scratch gave me a strong sense of accomplishment and made the development process especially enjoyable.

Installation Setup

The installation was constructed inside a room in my home.

I used tension-mounted curtain rails that could be installed without modifying the room. The curtains were positioned in the center of the room, and the screen was attached to the curtain rail.

To prevent the participant’s shadow from appearing on the screen, the projector was placed behind it. A horizontally mirrored image was rear-projected onto a translucent plastic sheet, allowing the projected visuals to be viewed clearly from the front.

The speakers were installed behind the curtains on both sides of the screen at approximately ear level. This arrangement helped create a more immersive sound experience.

Software

I developed a dedicated application using Processing.

As an initial prototype, I created a version in which pixie dust was drawn according to the position of the mouse cursor instead of the participant’s hands. I focused particularly on the movement of the particles to make the pixie dust appear magical and convincing.

After extensive experimentation, I developed the following behavior:

• The particles flicker randomly.

• After appearing on the screen for a certain amount of time, older particles begin to fall gently in an S-shaped motion.

I also implemented a painting mode, which allows participants to leave pixie dust at a fixed position by holding their hand over a designated area. Participants can also change the color of the particles.

Key Learnings

• Experience using a creative coding environment, which I found especially enjoyable

• Experience developing software with third-party libraries in addition to standard libraries

• Experience creating an interactive work centered on real-time graphics

Future Development

At the time of development, I used the laptop computer I already owned without giving much consideration to its processing capabilities. I also had limited programming experience.

Four years after creating this project, a wider variety of sensors and development environments became available to me, and my technical knowledge had grown significantly. I would like to revisit this concept and create a new interactive experience using the skills and technologies I have since acquired.

Gallery