"My Treasured Stories" is a digital interactive story projected onto any wall that the user desires. The story is accompanied by a treasure chest which the user interacts with the whole time. A toy of preference is placed into the chest before interacting with the story. This toy is then viewed as the main character of the story. Once the story is projected onto the wall, the user has many opportunities to choose how this story plays out.
An interactive digital story has been created to help children between the ages of 5 – 10 years old become more engaged in story time by providing an interactive sensory experience that focuses on playful learning where the child can personalise their story experience each time.
The product consists of a smart treasure chest, where a child can place any toy or object, they’d like into it. The toy is then scanned and inserted in as the main character of the story to be told & projected on to a wall. Here, the child can press a button to change a “page”, listen to audio, read the text on the wall or hold their hands in front of the ultrasonic sensor imbedded in the chest to manipulate different elements of the story in each scene.
Problem Statement
Traditional story books have become insignificant over recent years with the introduction of more interactive and fun digital experiences from technological devices. The fundamental learning skills and experience that a book used to provide is diminishing as a child now seeks more engaging, playful and imaginative experiences to enhance their learning, cognitive and emotional skills. User research showed that 9/10 times children preferred to play on a technological device, rather than read a book. Research also showed us that 8/10 times parents allowed their child to use this device.
Children nowadays use technological devices to keep themselves entertained, for example - like watching videos or playing games. However, they are now reading books less and less. It is here why we believe “My Treasured Stories” offers children between the ages of 5-10 years old an alternative to reading and learning. Our product is enriched with play, incorporating 2 concepts – digital interactive stories and a story bag. Combining both of these concepts have allowed us to modernise the traditional storybook experience. As a team we used research to draw from classroom Storytime concepts and integrate it into a household environment, where the child can partake in the experience by themselves, and still benefit. We have created a product which allows these children to interact, play and engage with a story while also learning at the same time. We have made sure that our product focuses on more than just literacy, “My Treasured Stories” improves the learning, cognitive and emotional skills of the child interacting with it.
Design Process
During the product design and build, our team focused on completing the must have features before working on the nice to haves. Our MVP had the bare minimum of the must have features for this to be a working product. Since iterating over the MVP, the product’s functionality has improved greatly. Functionality wise, we have a built treasure chest set up with the Arduino and breadcrumb board, led lights, web camera, pressure sensor, ultrasonic sensor and projector in it and working as desired; the story scenes, audio and text has been put together; an object can be placed into the chest, scanned into the projected story; the scanned object’s background is removed and the image is transparent for the story placeholder; there is manual scene change when the pressure sensor is pressed; story animations take place when a hand is waved in front of the motion sensor (max 30cm distance); the projector displays the story on the wall and the speakers play the audio from the story
Throughout the project, we worked with the 5 stages of design – empathising, defining, ideating, prototyping and testing. The first round of following this design process was used in line with a waterfall method between the team, where each step was followed, before the next step would start. After one round of waterfall, we had an MVP in place. After demonstrating the MVP to our cohort and collecting an abundance of feedback, we quickly started to adapt to an agile methodology where we worked in weekly sprints, improved different elements (at the same time) of the product from the previous sprint and solved demands that evolved through collaborative effort of self-organise and work in a cross-functional team. In the end we iterated over this design process many times, and since working in an agile team, our product outcome has far exceeded our original expectations of “My Treasured Stories”.
Technical Description
“My Treasured Stories” has 4 main functions when it comes to interactive code and the simulated functionality. Firstly, the chest can identify when an object is put into it and then scan that object to generate a 2D model picture. This is then placed it into the main character placeholder of the story in Unity. Secondly, the projector has been coded to start playing the story once this 2D model picture has been inserted into Unity. Thirdly, the touch sensor activates the scene changing and the scenario of reading a book aloud while setting up sound effects to enhance the sense of an atmospheric experience. Lastly, the ultrasonic sensor enhances the child’s interactive experience. When the sensor triggers movement, specific animations will play, enriching the playfulness experience for the child.
Techniques
The technical parts are separated into two parts. Unity is used to present stories. Each part of stories is built in different scenes and the audio of storytelling is also added to each scene to make the project more engaging. At the same time, we have realized the conversion of scenes in unity to simulate the situation of story turning.
We made a 330mmx260mmx380mm box. There is enough space in the box for children to put toys in, and all our technology is also reflected in this box. The first is the external webcam. This webcam can recognize the toy and upload the toy to unity in the form of a 2d picture.And we use API technology to remove the background, which can make the toy better fit the story scene. In this way, the toy will be presented in the story as the protagonist. This part of the code is programmed by C# for webcam. In addition, LED light strips will also be installed in the box to provide a light source for the toy to achieve the best shooting effect.
Arduino is mainly responsible for the realization of the functions of the two sensors.For scene conversion, we use a film thin pressure sensor, which can detect different pressures, and children can switch scenes by pressing this sensor. When the sensor feels pressure, the value will change. When the data is transmitted to unity, it will trigger a scene change. At the same time, in each story scene, we also set up different animations. For the triggering of animations, we use ultrasonic sensors. Ultrasonic sensors can be used to measure distance. When a child puts his hand within 30mm from the sensor, it will trigger animations in different scenes, bringing different visual feelings to the child.
Treasure Chest Build
Our box takes the form of a treasure chest. Unlike traditional treasure chests, our box has more height than width. This design decision was made based on user research conducted into the average size of toys belonging to children in our target demographic. The research indicated that the average size of the toys warranted a box that was approximately 380mm tall (not including the lid).
The majority of our box is constructed out of pine wood with MDF boards being used in certain aspects as it was thinner and lighter. The bottom of the box contains a slot that is used for organising cables and allowing them to lead out of the box without being too obstructive or noticeable. The box has a false bottom created from MDF. In addition to the false bottom, there is a contained space where all the technological components are stored. This space was created by using a piece of MDF to enclose an area of the box, creating a hollow wall. The components stored in this space include a breadboard with an Arduino, sensors, a camera, and LED lights. To allow for easy access for repairs and shifting components around, the MDF components are easily removable. The projector is stored in the box’s lid, to account for height and projection onto the wall properly.
After installation of the internals, the box was painted and detailed to complete the treasure chest aesthetic. The first step in this process was the application of two coats of brown paint to the box. After painting, a golden card material was cut up and glued onto the box. The reflective metallic gold contrasted against the box’s brown paint and ultimately made it look more like traditional treasure chests. Additional golden elements were installed to make the box look more aesthetically pleasing and to improve functionality. These elements include a golden latch to secure the closed lid and golden handles to allow easy transportation.
Final Statement
Our project focuses on the future mundane of storybook telling in a household by digitalising the users experience through a range of novel interactions which is utilised in future everyday technologies. The smart treasure chest scans an outside object (e.g. a toy) into the story as the main character and projects this story on to a wall of the house. Once projected, the story begins with audio in the background reading the storyline, animations on selected elements activated if the ultrasonic sensor is used and the story moving to the next page when the pressure sensor is pressed. Not only does “My Treasured Stories” meet the criteria of future mundane, it also incorporates our teams’ theme of enriched play by allowing for a combination of open and guided play. The playful aspect encourages users to be imaginative, creative, choose their own story and bring physical objects in to change the reality of the story.
Product Output
“My Treasured Stories” is a unique product that does not compare to any other products or research in the area. Currently there are an abundance of digital stories available to young children on technological devices such as iPad’s, androids and websites. Research shows some are more interactive than others and include multimedia (audio, wordless, touch interaction etc.), however most are used in classrooms only. There is no product that incorporates physical objects and the digital aspect together. Despite this, the common goal between our product and the rest, is the problem statement. Traditional books no longer deliver the necessary learning, emotional and psychological skills that children require. Parents and children turn to technology now more than ever, so it is imperative that there is a solution which can further grow the necessary skills but utilise technology.
User Experience
Our project’s desired outcomes were to create and build a product which enriched the play of children between the ages of 5-10 years old while improving their learning, emotional and psychological skills. Despite only testing our product with 2 children, we believe these desired outcomes have been met. One of the children, aged 7, told us that they felt more interested in the “My Treasured Stories” story instead of the traditional picture books they read. The other child, aged 9, said that the toys being brought into the story was a great way to make it feel like their own, ‘I got to use imagination to create my own story, interacting when I waved next to the box when I felt like it, otherwise I’d press the button’. We also asked some parents their thoughts on a pre-made story experience. The feedback we got from them suggested that their children would be more likely to have an increased attention span with all the multimedia while also learning how words are put together in a sentence. One parent said “I believe my child would use this interaction in their own stories they write about at school”. From this, our team believes that our project’s desired outcomes have been met.
