Nowadays, some people struggle to socialise and interact with other people at social events. These people do not want to initiate a conversation with strangers due to their fear of having an awkward and thus unsuccessful conversation. Spark aims to give users the confidence to initiate conversations by informing them how much they share interests, and acting as a conversation prop. Spark will ask users to enter their basic information and interests before entering the social networking venue. The information entered by the user will then be stored on the LED hat, and the user will also need to wear a necklace with a vibration function. When the user faces other people in a social activity place, the necklace will provide the user with vibration feedback. The stronger the vibration, the more similar interests between the two people. In addition, the hat on the user's head has LED lights, and the colour of each light represents a different kind of interest. However, the user does not know what kind of interest each colour represents. To discover the meaning of LEDs the user must have a conversation with others in the event, who would also be interested in discovering them. By allowing everyone to participate in a guessing game, Spark gives everyone a common conversation topic as well as motivating them to start conversations with others.
Technical Description
The Spark system consists of main application and the device, which communicate through WIFI. The main application allows users to enter their interests, process entered information, detect who each user is facing, and send relevant information to the devices upon receiving the request. The main application was implemented using python, using OpenCV and TKInter.
The UI of the application is used to enter user information to the system, which was implemented using python’s TKInter library. The UI consists of multiple drop-down menus and two buttons for submission and calibration. The drop-down menus are used to enter the interests of the user, which is the used to generate colour sequence and calculate vibration strength. The calibration button is used to calculate required parameters for coordinate conversion from 3D to 2D coordinate system, which uses a reference marker.
To detect who each user is facing, webcam and OpenCV library were used in conjunction with fiducial markers. The OpenCV library detects the location and orientation of fiducial markers in a video stream, as well as the transformation matrix between marker and camera coordinate system. This information is then used to map all markers into a 2D coordinate system. These converted coordinates are then used to calculate the angle between frontal vector and vector towards other markers for each fiducial markers present in the scene, allowing the detection of who is facing who.
The server run by the main application handles the communication between each device using web sockets. There were two types of communication between the server and the device: colour sequence and vibration strength. The type of communication is verified by the message sent by the device, and server sends relevant information based on the type of request and device ID. The server and communication are run on separate thread to main processing loop to prevent interference with the main application flow.
The Spark device consists of a microcontroller, Neopixel LED strip and vibration motor. Arduino core on ESP-32 thing plus development board was used as the central processor of the device due to its wireless connectivity and simplicity of implementation. Each microcontroller has hard-coded participant ID, which is used to update its LED colour and vibration strength. When microcontroller is booted, it first connects to the application server via wifi and requests the LED colour corresponding to the device. Once the LED colours are received, the microcontroller will update the colours of LEDs to match the received data. Then, the device sends periodic request (4 times per second) to update its vibration value, which is then used to control the vibration motor.
The microcontroller circuit and battery of the Spark device are attached inside a hat with padding, which is sewed inside a fabric hat. The LEDs are attached to the front of the hat, and vibration motor is attached in the form of a necklace. The hat also has fiducial markers on cardboard attached on top of it to ensure the functionality of marker detection.
Final Statement
The exhibit was a great experience for the team to not only present and explain our idea to a diverse audience but also to receive feedback about our concept and prototype. Our team was the first team to present in front of tutors and we quickly ran into issues with our prototype. In detail, one of the wires for the vibration motor snapped off which caused the necklace to not vibrate when facing a person with a shared interest. Fortunately this only occurred to one of the necklaces so the demo was able to proceed with only one of the necklaces working. However, it became apparent that our lack of user testing and how fragile the physical prototype itself was. The exhibition became a great place for us to user test and also receive feedback.
The diverse audience members allowed us to receive feedback from the exhibition which we could implement into future directions for this project.
One of the major pieces of feedback we received was how this idea could be implemented into real-life work scenarios. For instance we had a staff member from Suncorp explain to use how throughout the COVID pandemic staff members would wear different coloured lanyard to indicate whether they would like to be approached or not. He explained the difficulty of the lanyards being that sometimes it would be hard to physically see the lanyard. He further explained that with the technology used in Spark, staff members wouldn't have to wear different coloured lanyards but just face the person and figure out via vibration if the person wants to be approached or not.
Another insightful piece of feedback the team received was that we may have not fully considered how the physical appearance of a person effects the approachability of a person. Specifically, the audience member illustrated that sometimes people are very judgemental of physical appearance of another person and decide whether they would be interested in a person purely based off their appearance. It is clear in our design that we did not fully consider this as we were too focused on the environment being used in an ideal environment that people would not judge other by their looks but by their character.
