We are using a combination of Unity and Arduino, simulate the
natural environment of the ocean and show the creatures, landscapes
and phenomena. In order to bring a good experience to the user, this
project allows the user to operate gloves and a stress ball to
control the tuna in the display.
The main equipment includes gloves, stress balls, and a display
screen. The glove is equipped with a gyroscope to detect the
inclination of the hand to control the direction of the tuna. The
fingertip part of the glove is equipped with a pressure monitor to
detect the pressure of the fingertip, so as to simulate the feeding
behavior of tuna. A pressure detection device is installed inside
the pressure ball to detect whether the pressure ball is under
pressure, so as to make the tuna move forward. In addition, there is
a vibration device inside the glove to alert the user if the tuna
has touched other entities in the scene.
When the project starts, users need to put on gloves and hold the
stress ball in their hands, and then they can see the scene
simulation of the underwater world appear on the display screen.
When users want the tuna to advance, they need to squeeze the
pressure ball with their hands, and each squeeze can move the tuna
forward a certain distance. When users want to turn the tuna, they
just tilt their palms. Leaning to the left means to turn left, and
leaning to the right makes the tuna turn to the right.
After manipulating the tuna close to the prey, the user needs to
touch the fingertips of both hands to generate enough pressure to
make the tuna eat the prey. When the tuna touches other objects in
the scene, such as stones, the vibration device in the glove will
vibrate to remind the user.
Technical Description
The original prototype consists of a marine environment in Unity and
a physical model with sensors on Arduino circuit playground or
external. The connection between Unity and Arduino board is realized
by Ardity. The specific sensors and technologies used are as
follows:
1.Arduino circuit playground's own accelerometer
controls the turning of fish (as before). Change to one-handed
control.
2.Flex sensor controls the way the fish moves
forward. Flex sensor is similar to bone, which connects users' hands
and wrists and can sense their activities. The user's hands bend
left and right once in a group, and each group will control the fish
to advance a certain distance. The speed of left and right bending
determines the speed of advance. This way can not only vividly
simulate the movement of the fish, but also control the swimming
speed of the fish.
3.The press sensor located at the
mouth of the toy fish controls the way of eating. When the user
squeezes the fish's tongue with his finger, the fish will perform
the action of "eating".
4.Unity scene adds pop-up prompt
box to remind children. When fish eat other low-level creatures, the
pop-up window will tell them what they just ate. When the fish
touches the shark, the motor inside the toy fish will vibrate to
remind children that it is a dangerous creature. When children try
to eat a shark or are about to be eaten by a shark, a pop-up window
will appear and warn them, but the system will not show the process
of fish being eaten. This picture and experience is unpleasant for
children. We hope that children will learn about marine life in a
relaxed environment and have more positive thinking.
In
addition, the fish controlled by children will vibrate when they
touch the shark, but the pop-up window will not appear. They can
decide whether to leave or eat the shark by themselves. Only when
they try to eat the shark or are about to be eaten by the shark will
the pop-up window appear and warn them. If a child sees the fish he
controls being eaten by a shark, it may make the child feel sad.
Therefore, we try to prevent this from happening by directly
informing them. Although we give children tips and warnings, we
still hope to give them the opportunity to explore and try on their
own initiative.
5.When the fish eats the prey, the
microphone that comes with Arduino circuit playground will play
pleasant sound effects to cue the child a proper predation has been
completed.
Final Statement
Exhibition experience
On the day of exhibition, we were so excited that we could show our
product to the public. We set up our place by 12:00pm, there was
something wrong with our projector, but luckily our tutor brought us
a good one.
We also prepared some snacks for the audience.
After 1:00pm, we got our first audience, they were impressed by our product and played for a while. And then, our teaching team came to experience and mark our product. After marking, we basically welcomed many students audience to experience and ask them for suggestions.
But, our flex sensor was broken, since someone treated our Dory fish so harshly, we then decided to fix this during the break before 5 pm.
From 5 pm to 7 pm, we welcomed our photographer first.
Then, there were many professionals or people from industry visited us and provided many valuable suggestions.
After 7 pm, our tutor announced the prizes for some groups, congratulations to them! Before 8 pm, we packed up all the settings.
Our team enjoyed the whole process of the exhibit since it gave us a chance to get some valuable feedback from professionals and learn how to deal with emergency situation since we needed to fix the circuit.
Feedback & Response
In general, the evaluations we received from the teaching team and audiences are positive and constructive. The visual effects displayed in the prototype, the simulation, and the natural role-playing and control methods received a lot of affirmation and appreciation. In the interaction with virtual reality, the synchronized vibration and sound feedback of the device in hand also surprised some audiences. During the exhibition, we also received valuable feedback, which mainly focuses on the following aspects:
1.The requirement for more interactive content in the visual scene. The current prototype can only demonstrate the most elemental functions, including an edible jellyfish and a shark that cannot be preyed upon. Other fish, plants, hills, lights, and bubbles in the scene serve more as aesthetic visual effects or pictures than as entities with interactive functions. This demand is reflected in the fact that the majority of the participants at the exhibition, after completing their interactions with specific jellyfish and sharks in practice, usually tend to look for other objects in the scene, such as schools of jellyfish or seaweed, to perform the action of predation. Secondly, the collection and reward mechanism of the game itself was also mentioned by individual viewers. For example, if the glittering pearls in the scene can be added to the prototype gameplay design as collectible props or achievements, it is likely to motivate our target group to engage in interactive activities and explore the virtual world.
2.Functional upgrades to suit the workplace. One of the professionals who coined for us highly praised our prototype and offered ideas and some possibilities about specific application scenarios. He believes that our concepts and products are suitable for use in museums as a means of popularizing marine knowledge. In this case, The physical interaction scenario of the product will expand from a small place to a hall or exhibition room. A wide venue means more individuals can be involved in prototype interactions as well as group activities and collaborations. After a group discussion, we believe that this idea is more relevant to the proposed concept than the original implementation, which can be taken as a direction for the future development of this product.
