Music Masher

Our project mainly aims at assisting children's musical exploration stage through the form of a cube that interacts with different movements and different environments to produce different sound effects. Through a new model of musical interaction, different sounds are allowed to be delivered through a single medium, thus increasing the diversity of sound combinations. At the same time, the user can be inspired to explore the creation of music and expand the medium of the traditional music learning and cognitive process.

Our design can be placed in any environment. It can be thrown and struck, and the cube produces different sounds as a result and the volume of the sound transmitted is also different; while in use, the cube will also produce various forms of LED lighting effects. In addition, the visualization of sound effects through LEDs is our focus. According to survey research, music visualization can stimulate children's curiosity for music to a greater extent and help them better remember different musical notes. At the same time, we designed special ambient music properties. When children place the cube in different environments, the LED visualization light effect can also help them better understand the surrounding environment and help them be motivated to learn music.

Technical Description

1. Hardware

Arduino M5stickC

At the beginning of the design we chose to use m5stack C as our hardware. Since M5stickC has built-in accelerometer, it can give different feedback according to different position or displacement. So we're going to use the accelerometer to make the cube make different sounds at different positions. Or switch between different tones or beats by shaking. The wireless connection is also one of the reasons why we chose it, because it allows users to use their imagination as much as possible to generate more interaction with the cube. However, due to the limited sound that the speaker of M5stickC can play, it cannot meet our design requirements. So in the follow-up development we replaced the Arduino hardware.

Arduino UNO

The Arduino Uno is being utilised to compile the data received from most of the sensors with the acception of some data sent to the serial via a lilypad. This enabled us to compile up to five cubes and process them to be visualised and sonically created.

Arduino Lilypad Plus

The lilypad is being used sometimes in place of other sensors to record different inputs in a more stable form such as temperature, accelerometer and gyroscope. Other sensors used will be connected through the Arduino UNO.

2. Software system

The software utilised for the project consists of Processing for the creation of the visualisation of the cubes and the music beat mapping for each of the Arduino inputs, to enable this connection the Arduino IDE assists in reading serial data and sending this data to processing, music files are also imported into the processing folder structure. The software runs the output in a view window and this is used as a projection from the laptop to visualise to the users what the interactions look like within each of the cubes.All the data is read between Sensors, Lilypads and processing

3. Product main body

The body of the product is a cubic frame created with 3D printing technology. We reserved enough space to install the necessary components inside, including the Arduino we used, the Led strips that light up and change the color of the cube, and the power bank to provide energy.

4. Visualisation

In order to allow users to have a better visual experience, we project the visible system animation effects from the computer and projector onto a large white screen.

Final Statement

Exhibit Experience

Overall the exhibit experience was a beneficial end for the project. The space that we were able to create with the projector and cubes, as well as a auditory environment was enriching and enabled us to convey the intended experience effectively. One slight set back was the ability to see the visualisation during the day but this was alleviated come night time. The experience itself definitely became more enchanted in the later hours of the exhibition, as the lights of the cube and projection became more prominent.

Audience Feedback

During the exhibition, we invited many people to experience our design product and introduced them to the concept and function of our product. These visitors think the design concept and function is useful and exciting for children to explore music, but some people also made suggestions. For example, could two cubes interact with each other, such as two cubes collide or rub each other? Can two cubes play simultaneously and have different instruments' sound designs? Can the LED Light effect be changed by increasing the strength or speed? In addition, we also invited children to experience our cube in the exhibition. During their use, we observed that children attract by the images in our projection and started to adjust the Cube, shake it hard and twist it, so we think this visualization is an excellent way to attract children.

Functionality

The prototype we used in the exhibition was still a little different from our original idea. We originally wanted to implement wireless remote control of the product, which would give users more space for interaction. However, the stability of signal transmission is a more important factor to ensure product use, so for the prototype we still had wired signal transmission. In addition, we originally wanted to allow users to choose their favorite songs to play. But after replacing the music files, the program cannot run normally due to file format problems. We regret not being able to do more testing before the exhibition to find this out. Other functions worked fine and remained stable throughout the day, this enabled the widest possible audience to experience and understand our products and concepts effectively. We are proud of this outcome.

Future directions

For the future direction of our project, we will first focus on improving the user experience in our human-computer interaction. Our project requires the cube to be connected to the computer to display specific visualization effects, which highly limits the space and venue for users to use this cube. Therefore, we will use Touch Designer to improve the user experience in the follow-up. Touch Designer can ensure that users are not limited by venues and equipment when using our product and can also enhance the visual effect of visualization to a greater extent. Secondly, we need to improve the product's wired transmission to wireless transmission and improve the wireless stability as much as possible to cooperate with the touch designer to perfectly present visual effects so that users can experience our products more immersively. Finally, we will also add more songs to give users more choices.

Image Reference

eventphotography.com

Video images as referenced in Youtube description, all other images self-produced.