From record players to mobile apps, the way of music sharing has been changed from physical to virtual. Smart speakers have brought screen-free interaction, and the mental detection AI technology may help to sort out music based on users emotions, so that we can share our feelings effectively and efficiently through music. The future music streaming tends to be immersive, collaborative and artificial intelligence. In a fast-paced work environment, people have suffered more mental pressures than before.Therefore, we want to build MoodBall for people to communicate moods through music and connect users with their families and friends in everyday life.
MoodBall is an emotion detection ball that allows users to detect and share feelings through emotional ways including music , lights and other visual feedback projected by other devices. It aims to create immersive music modulating experiences for joy and relaxation as well as to provide mental support through sharing and engaging with remote users. A user can interact with the ball through touching, shaking, rotating and pressing, and the ball will be able to play relevant pieces of music that reflect the user's mood. The ball can connect to other remote balls to share a user's current mood with others. When the remote ball receives the data, it can play the same music with lights and visualization. We choose to express and share emotions through music, lights and visual feedback instead of words since the former three types are emotional rather than logical. Using emotional ways to measure emotions is more precise than using literal words. Particularly, introverts are usually ashamed to express themselves through words, but they can tell a story and share how they “really feel” through emotional ways, which allows more people to “understand and empathy them”. Furthermore, the ball can connect to other playful devices like the projector to provide visual feedback that further expresses the user's mood. The richer the emotional ways are used, the more precise the mood will be measured. Lastly, the ball also has some abilities to detect the degree of emotions. For example, the faster the user rotates the ball, the faster the tempo of the music will be performed and the richer particles will be given in the visual feedback.
The view of "music not merely communicates basic emotions, but related to constructivist account" from Cespedes-Guevara[1] has inspired us to explore how music expresses core emotions and how to influence listeners' mental status. Our idea of developing concepts from the perspective of music expression and emotion detection was inspired by "Solo" Emotional Radio[2] that can recognize the user's facial expressions and play the music that matches the user's mood. The sensors used in an audiovisual installation[3] that can translate emotions into light inspired us to build the emotion detection function. The several interesting physical interaction methods of our device to convey user emotions were inspired by the mood reflecting floor that detects emotion by different body language such as fast and powerful movement.[4]
Technical Development
Emotion detection systems include GSR sensor, pressure sensor and accelerometer gyroscope, and each sensor’s level is based on the average rate from user testing responses. According to the users' responses, we found some common trends in which different users interacted with the ball under different moods.
Touch the ball: calm (5/9)
Shake the ball: energetic (5/9)
Squeeze the ball: angry (5/9)
Roll the ball: happy (5/9)
We used accelerometer to measure the acceleration and rotation of the ball that indicates the rotation and shaking behaviours while using two pressure sensors to measure the pressure forces represented by the squeezing behavior. Since touching and squeezing are similar, we added the GSR sensor to detect the strong emotion and the hand movement by utilizing its slope reading.
Music and visual feedback: Speaker and projector are used to build an immersive music playing experience. Different background music, light colors and visual feedback provided by projector animations represent different moods.
Calm: music (Calm), light (Blue), visual (Water Wave)
Happy: music (Happy), light (colorful RGB), visual (Colourful Heart)
Angry: music (Angry), light (Red), visual (Fire Burning)
Energetic: music (Energetic), light (Green), visual (Green Leaves)
Link to the coding files and technique documents:
Pressure Sensor
GSR Sensor
Accelerometer Gyroscope
Combined Sensors With Formatted Outputs
Unity for Receiving Formatted Outputs
Unity for Switching Backgrounds
Background music
Video prototype
Link to other graphic design documents:
Form & Function
1.Physical & Playful: Interaction with touching, shaking, pressing,
rotating, that creates a novel approach to share emotions
2.Detect Moods: Detect users moods based on the physical
interactions with the ball.
3.Modulate Music: Use sensors' values indicated by different moods
to play different pieces of music.
4.Share emotion: Send the piece of music and lights to a remote
ball.
5.Remote Interaction: The remote ball is able to receive music and
lights
6.Future Mundane: The affordable materials, the portable size and
the daily emotional shaing make the ball possible to be a future
mundane product that can be used by family members and colleagues.
Final Statement
The construction and preparation went very well and all the sensors worked. The only pity is that the sunlight was too bright outside, which makes the projection effect not as good as the indoor scene we set.
At the beginning of the exhibition, participants often asked or observed how others use the ball before trying it because they are confused about how to interact with it. We immediately wrote down the steps on our blackboard and explained them to participants at each beginning. In this way, they understood the process better.
After that, it went quite well in terms of participant experience and response. Participants are often surprised and interested by MoodBall's reaction. Especially at night, MoodBall attracted more participants and brought them an excellent immersive experience including LED lights, pleasant music and beautiful visual effects. What we did not expect is that MoodBall attracts a lot of children. We are so glad that MoodBall is so popular among different age groups. Generally speaking, the device has reached our expected user experience.
There are also some deviations in the data set. For example, it's sometimes difficult for children or girls with less strength to trigger angry mode. In calm mode, the setting of press sensors is high, which makes users squeeze rather than touching the ball. That’s why they cannot distinguish between calm and angry modes sometimes. Additionally, we initially let users shake the ball back and forth or left and right to trigger the energetic mode, but when they were told to shake, most of their reactions were to move up and down. These feedback helps us understand user behavior and needs better and contributes to MoodBall’s success in the future.
Overall, the product was well received by the guests and the teaching team. Most of the guests thought MoodBall interesting and said that the concept met their needs to a certain extent, for example, by using music and lights to communicate their moods to their close friends and family in a more visual way. At the same time, some users said that the projection made the whole product more appealing rather than just using music and lights. In addition, some users also provided us with constructive comments that could improve our product in the future. For example, he would like our ball to be more flexible, so that he can interact with it in a more intense way when he is angry, such as throwing. That also confirms the results of our user testing.
Due to the limited time, the remote function is not implemented enough and has less attention in the exhibition by participants. In the future, we can let users comfort each other remotely. Different types of music can be updated automatically in case users are tired of them. The material of the ball needs to be more bouncing and flexible to let users express their emotions freely. We believe after these developments, MoodBall will be a real future mundane product and popular by lots of people.
Reference
[1] Cespedes-Guevara, J., & Eerola, T. (2018). Music Communicates
Affects, Not Basic Emotions – A Constructionist Account of
Attribution of Emotional Meanings to Music. Frontiers in Psychology,
9, 215. https://doi.org/10.3389/fpsyg.2018.00215
[2] Guardian News, Solo, the “emotional radio” that plays music to
suit your mood. 2017. https://youtu.be/-8nVJ3Zz_SE
[3] Dezeen, Audiovisual installation translates emotions into beams
of light. 2017. https://youtu.be/8FkBA3xTne0
[4] Visualcraft, The mood reflecting floor - Interaction design
project. 2014. https://youtu.be/JOg4chJAgn8
The materials we looked for:
