Inspired by the tradition of having conversations around a campfire deep into the night, ‘Art Through Heart’ is a campfire-shaped light fixture that functions as a conversation aid. It aims to help people form deeper connections with loved ones by promoting awareness of another’s emotions and enhancing feelings of empathy during a conversation, as well as allowing people to create beautiful mementos of the emotional beats of their conversations to be experienced again at a later time, which, unlike voice and video recordings, are beautiful representations of the emotions felt and how they fluctuate throughout a conversation.
The campfire reflects the emotions being expressed by interlocutors through the colour of the lights, which subtly change with the mood of the conversation. Additionally, as research has shown that seeing a representation of another person's heartbeat while they are talking increases one’s capacity for empathy, the device detects the heartbeats of interlocutors through mugs that they hold, which are then reflected through the gentle pulsing of the campfire.
Technical Description
‘Art Through Heart’ comprises three major parts: the main ‘display’ component, the campfire-shaped light fixture; the ‘detection’ components, the mugs; and a web application, which acts as the transport hub for events which are sent and received from the mugs and campfire. It is also used as a user interface for the users, where they can start the conversations, and access and replay the mementos generated for their conversation.
Mugs
The mugs are composed of a Raspberry Pi Zero, a small USB microphone, and a heart rate sensor, which are all contained within a two-part 3D-printed housing. The Raspberry Pi Zero, which sits in the bottom of the outer part of the housing, sends data via a Python script with Socket.IO to the webserver.
Campfire-Shaped Light
The campfire-shaped light is composed of laser-cut and etched acrylic panels, a 3D-printed base, Neopixel LED strips, a Raspberry Pi Model B+, and a button.
The 3D-printed base features channels sized specifically to accommodate the Neopixel LED strips and the acrylic panels in a way that allows the light to most effectively shine through the acrylic. This base was raised up using wooden legs to create space for the Raspberry Pi 3B+, which receives processed events from the web server to display the appropriate light colours for the emotions received using a Python script. This was slid into a cardboard housing featuring a slot for the button, which turned the Neopixels on and off while also starting and stopping the recording and processing of the conversation.
Web Application
The web application is built on top of the MERN stack, namely MongoDb, Express.JS, Node.JS and React + Typescript. Socket.IO is used as the framework for realtime, bi-directional communication between the devices. This application makes use of the IBM Watson Tone Analysis API and Speech to Text API to process and determine the tone of the conversation users have between each other.
Final Statement
Exhibit Experience
Despite our concerns about exhibiting our light-based concept outside during the day, we were able to create an environment dark enough for visitors to see the colour of the campfire and had a large monitor to allow visitors to see the web application in use. While visitors were still able to see the lights during the day, the concept was most effective after the sun had set, as the light from the campfire made the whole exhibit glow.
Functionality
Our original concept involved a gentle pulsing of the lights in time with the heart rates of the users; however, due to time constraints, we weren’t able to implement this without causing issues with the displaying of the colours according to emotions. However, we were still able to show the placement of the heart rate sensor in the mugs and explain their purpose.
The day before the exhibit, we had been concerned about how effectively the microphone would be able to pick up the voices of visitors in the context of the exhibition. However, visitors were able to hold the mugs at quite a fair distance from their faces without causing any issues.
For most of the exhibition, the campfire was able to be controlled independently (i.e. starting and stopping conversations) via the button on its side, with the web application only required when choosing and replaying a memento. Due to the repeated use of the button, it broke towards the end of the night; however, this was not a major issue, as the web application could be used to perform the same function.
Reception
During the day, we did not have many visitors to our exhibit; this may have been due to both the small number of visitors to the Interaction Design Exhibition as a whole, as well as the fact that it was difficult to see the lights in the daylight. However, there were many more visitors during the evening, as the glowing of the campfire was more effective at drawing visitors’ attention—there would often be larger groups of people watching from the edge of the tent while a visitor was interacting with the campfire.
Many visitors complimented the appearance of the campfire, with a few stating that they genuinely wanted to have it in their homes. Additionally, all of the visitors were enthusiastic and amazed at both the concept’s ability to relatively accurately display their emotions as well as the processes behind the display, especially the visitors who had a keen interest in the technical aspects of the concept.
Key Steps for the Future
Future iterations of the concept should focus on implementing multiple mugs and the currently-missing heartbeat functionality. Additionally, while the concept was able to accurately display emotions from a person’s speech, it doesn’t take into account the speaker’s tone of voice; thus, the analysis of speech quality could also be implemented. Lastly, other contexts in which the concept could be used, such as aged care homes, should be researched.
