Our project is an interactive musical installation that connects the world of music and the real world. It is primarily designed to help newcomers who want to create music but lack the relevant production knowledge (mainly acoustic effects) to explore the world of music, understand the function of different instruments in music, and understand the use and operation of acoustic effects.
Our project consists of four cubes, representing different instruments: bass, drums, guitar, and piano. The six sides of each cube correspond to five different styles of playing loops for that instrument, as well as a mute side. When the user places the face of the cube on it, the corresponding audio loop is played. The user can rotate the cube to switch between the different audio for each instrument, as well as understand the role of each instrument in the overall audio.
Also, our project contains three physical interactive devices that allow users to explore acoustic effects such as reverb/echo/equalisation and gain a deeper understanding of these effects by interacting with the devices to change the sound field (e.g., sound field size/sound medium).
In addition, user testing and feedback indicated that the auditory feedback of the cubes was a bit monotonous for beginners. Therefore, we added LED lights for visual feedback. The LED lights change according to the volume of the music, thus enhancing the user's understanding of the music. The volume is softly accompanied by soft music.
Technical Description
Hardware
Arduino Pro Micro/ Arduino Leonardo
These boards use ATmega32u4 which supports native USB MIDI plug and play functionality and keyboard functions. These were used to create a MIDI interface with MPU6050， ADXL345, potentiometers and arcade buttons.
Arduino UNO
Used to control the Frequency sensitive LED matrix with digital LEDs and analog sound sensor.
Slider Potentiometer
This gets +5v and GND and outputs a signal
MPU 6050 module
This module outputs the acceleration, temperature and rotation values (rads-1) with SCL and SDA pinouts
ADXL345 module
This module outputs the acceleration and rotation values (rads-1) with SCL and SDA pinouts
Arcade Style Push buttons with LED
WS2812B Digital LED Strip
This is a programmable Digital LED Strip that can be programmed with FastLED.h libraries. can be controlled with just one pin out from a development board.
Analogue sound Sensor
This sensor gets a +5v and GND and outputs an analog sound signal from another pin.
Software
Ableton Live is a powerful tool through which we have edited/created the audio we need. We connected the Arduino to various buttons/knobs/gyros, sending MIDI signals to Ableton Live and assigning each button to the parameter to be modulated via MIDI mapping. We also used Ableton Live's Session View Clip to assign multiple loops of different styles to each instrument track, triggering the corresponding loop when the cube was flipped.
