Project Description
Through research interviews, we find children are difficult to learn musical notation and easily give up. Thus, we try to find out how to help them learn musical notation and encourage them to play piano. Our intended concept is to design a HAPPY PIANO for children aged 4-6. This piano consists of a smart piano and a 3D holographic projection. The piano is used to help children learn musical notation, when a user sees the note they do not know while playing the piano, then clicks on the note on the booklet and the corresponding piano key on the piano lights up. Moreover, there is a children’s favorite character in the 3D holographic projection that is used to accompany/encourage/teach children. When the user plays the right piano key, the corresponding note on the booklet turns green, and if the wrong key is played, a role in the full range projector encourages the user to cheer up. When a user finishes one song, the character will celebrate him/her. Children could choose their favorite characters with a button. Although there are many similar products on the market, such as the iPad, which has software for learning piano score, electronic products are harmful to children's eyes, so this design can also play a role in protecting children’s eyes.
Technical Description
First of all, we have to make the structure of the piano. We use laser cutting the acrylic sheet for the production of piano materials. This is a very important part in making the piano. We measured the real proportions of the piano keys and geometrically narrow it, such as white keys, black key steel, piano appearance, etc., and drew and printed them in CAD.
Putting the Arduino inside the transparent piano, we chose to use mechanical principle to connect the Copper sensor and control the internal mechanical parts by pressing the keys. Rubber bands are used to fix the keys and mechanical parts to simulate the feel of real piano keys.
Inside the piano structure, we currently use two Uno boards. In one of the UNO boards, a pressure sensor was used to link each note in the music score corresponding to the LED light belt on the key. We also used the Copper sensor, wrapped the Copper sensor on the back section of the piano key, and placed a longboard at the bottom, and wrapped the Copper sensor on the board. The ground end of the other UNO board is connected to the longboard, and the remaining keys are connected to different pin ports, which are written in the form of switches with codes. When the Copper Sensor on the keys contacts the long board connected to “ground”, some operations can occur. We used Unity and Arduino interconnection to achieve the piano sounds, as well as control the corresponding note lights on the score. In this way, the target user can check the music score practiced before, so as to achieve the purpose of learning to recognize music score. Due to the limited number of Arduino Uno pins, we were not able to achieve all key interactions.
Moreover, we drew the hologram pyramid with CAD and printed it out. During this process we considered using a thinner transparent plastic material for printing, but in the end we chose acrylic because it is stronger and more transparent, and it produces a better holographic projection. Then we found out that the transparent acrylic could clearly see the messy lines inside, so we made the keys and the box where we put the score into a frosted pattern.
At last, we used Unity to create four different 3D characters and animations of them in the holographic projection. Also used the button in Lilypad to transmit data to achieve the effect of role switching. Then, we used Unity instead of Ableton Live for the sound effect of piano keys.
Final Statement
Exhibit Experience & Public Response
During the Exhibition, there were a lot of people visiting the exhibition, and we invited some of them to experience our project, and described the concepts and features of our project to them. The response from these users was that they thought the concept and functionality of the project were good and interesting, and some of them made some comments which we noted down. For example, 3D holographic projection could be set on the piano; Place the corresponding key lights on top of the keys rather than inside them; Put the teaching guide notes on the left and the practicing music sheet on the right, the purpose is to distinguish regions to avoid confusion among users; Put the button for switching 3D roles on the operating table of the piano instead of the back of the piano, because it is inconvenient for users to switch; Visualize the sound of each key of the piano, for example, showing the rise and fall of each note, and so on.We also asked some of the users some questions, such as the problems and difficulties when they are learning to play the piano. Most of them are as easy to give up and get bored like the users we interviewed before. Moreover, in the exhibition, we also invited some children aged 4-6 to experience our pianos, hoping to continue to improve our project through their reactions. First of all, children will be attracted by the character in our 3D holographic projection and then start to use the piano, so we thought that this holographic projection with cartoon characters would be a good way to attract the attention of children. In addition, we observed when users use our pianos, it is very difficult for them to be distracted from watching the changes of the animation when they are learning the piano.
Therefore, we hope to give the animated characters more interaction in future projects. For example, in addition to changes in the characters' movements, interactive guidance of sound can also be added. Animated characters can demonstrate how to play the piano; When the user plays the right key, the animation character will be applauded, playing the wrong key, the animation character will be encouraged to the user, and so on.
Future Work
The next step for our project is to add the toggle function of music booklets, which means users can prepare their favorite music score and put it on the piano rack. when they do not know any piano note, then directly click that note instead of a fixed music score. Users can change the booklets according to their own learning level. In addition, our team believes that by combining these user feedback, we put the button for switching 3D characters in a position where it is easy to switch, such as next to the box that covers the holographic projection.In the end, we hope that this project can really help our target users through continuous improvement.