Summary

In Japan, “kamishibai” is a form of storytelling in which pictures of a story are shown along with the performer’s narration. However, often, the children were only listening to the performer. Thus, the kamishibai lacked elements to allow children to enter into the story. In this system, children are given a special picture-book device that outputs light, sound, vibration, and aroma effects. The device on which the narrator places the picture and the devices held by children are constantly connected via wireless communication. Each device performs in synchronization with the story scenes. Therefore, children are invited to enter the world of the story. My work was shown to the public at an event and I saw many smiling faces.

Purpose

• Create an interactive storytelling experience that brings the audience into the world of the story through their imagination.

• Invite the public to experience the work imagination product.

Background

Little Hopper, a digital fabrication school in Yokohama, planned and organized the exhibition of Glowing Picture Books. The Glowing Picture Book is a device that allows readers to enjoy stories in the dark by fitting an acrylic board printed with picture book pages into an A4-size wooden frame with built-in LED lights. At the exhibition, Glowing Picture Books created by children were displayed throughout the venue. This time, I collaborated with this project and developed a storytelling event using Glowing Picture Books.

Process

Sketch

First, an image of the event was created for the overall planning. The style was similar to a typical kamishibai session, with participants gathering around the narrator to listen to the story. Glowing Picture Books are products designed to be used in the dark. Therefore, the event was also to be held in a low-lit space. Next, I considered ways in which participants could enjoy the storytelling. The concept of " immersive storytelling" was determined based on the idea that the most exciting thing for children participating in storytelling is to be able to step into the story. To achieve this goal, I decided to hand out to each participant a special device that would perform actions in synchronization with the narrator’s Glowing Picture Books and provide a sense of immersion through special effects like sound and light effects.

Required Specifications

The specifications of this system were determined as follows.

• The parent device (narrator’s Glowing Picture Books) and the child device (picture-book device) are connected via wireless communication.

• The parent device recognizes the ID of the set page and sends the scene number to the child device.

• The child device changes its performance according to the data received from the parent device.

• The child device will have a mechanism that enables the following performances.

• Lighting effects (front cover and inside of the picture book)

• Vibration

• Scent

• Sound

System Configuration

The parent device communicates wirelessly with a laptop PC and four picture-book devices. The laptop PC controls the background music and the image projected on the ceiling by the projector.

Development of Hardware

A parent device and four picture book devices were developed. The devices were produced by cutting and assembling MDF boards with a laser cutting machine. Each page has an NFC tag, and the parent device has a sensor to read them. The picture-book device is a spread type and has a constellation pattern engraved on the inside.

Parent device

Child device

Consideration of Electronic Devices

As a first step, prototype electronic circuits were built on breadboard and stripboard to verify the devices used, confirm their usage and confirm that the intended performance was possible.

Electronic Circuit Board Design

I designed printed circuit boards (PCBs) and ordered them. After that, electronic components were soldered on the board. Two different PCBs were designed for the parent and child devices. The required devices can be controlled on a single PCB. All devices, such as LEDs and speakers, are connected by removable connectors so they can be replaced in case of breakdown. An AC adapter provided the power supply for the parent device, and the picture book devices were powered by a 9V battery that can be recharged and used repeatedly.

Scenario Creation

I received the pages that will be read at the event from Little Hopper and discussed the performance details. Six parameters can be changed: (1) LED color, (2) sound effects, (3) vibration, (4) scent, (5) background music in the room, and (6) images projected on the ceiling ((1) to (4) are for picture-book devices). Suitable combinations of effects for each page of the story were considered.

Software

According to the scenarios considered, coding was done so that each device would operate according to the recognized page. For example, when a page of an ocean scene is set on the parent device, the LEDs on the picture-book device glow light blue, and the sound of bubbles is played from the built-in speaker. Also, the device vibrates with the sound of bubbles, and the sensation is transmitted to the reader’s hands. The sound and vibration are output randomly, and the behavior differs for all four devices. In a tea party scene, the device outputs the sweet aroma of cake. Coding for the parent and picture-book devices was handled by the co-producers.

Operation Tests

Electronic circuits and programs were implemented in a picture-book device, and its operation was verified.

Performance at the Event

The Glowing Picture Book exhibition ran for five days, and two performances of this work were given each day. Four families participated in each performance and enjoyed the world of the story. All performances were completed without significant device problems.

Experiences

• Experience in making electronic circuit boards with printed circuit board (PCB).

• An experience that allows the public people to experience the work.

• Experience in providing immersive and interactive entertainment.

Related Web Sites

• Crowdfunding for the Glowing Picture Book exhibition

  「電気を消したら、遊びがスタート」光る絵本の世界を子ども達100人と実現したい！

• Little Hopper, the children's programming and digital fabrication school

  子ども/小中学生のプログラミング・ロボット教室 | リトルホッパー | 都筑区

Gallery