Hand-2-Hand

by Among Balls

For when you need an extra "Hand"

Hand-2-Hand is a physical computing project that bridges digital and physical communication by transforming virtual interactions into tangible experiences. Using robotic arms, our system enables remote users to physically interact across distances, creating a new dimension of collaboration and emotional expression.

The system consists of paired robotic hands that mirror movements and gestures in real-time. When one user moves their robotic arm flex and pressure sensors capture these inputs and transmit the data to a corresponding robotic hand at another location, which replicates the exact movements. This creates a sense of physical presence during remote interactions, allowing people to "lend a hand" across any distance.

What sets Hand-2-Hand apart is its approach to data physicalisation, transforming digital gesture data into physical movements that can be physically experienced. Users can physically interact with the data, like responding to a fist bump given from the other end. As an enhanced collaboration tool, Hand-2-Hand addresses the lack of physical connection in remote work enabling intuitive collaboration for tasks requiring manual demonstration.

A paper prototype of a mechanical hand is shown, with strings attached to each finger. When the strings are pulled, the paper fingers lift, mimicking real finger movement. A person's hand is holding and pulling one of the strings to demonstrate the mechanism. A person demonstrates a 3D-printed mechanical finger connected to strings and powered by a microservo. The setup is wired to an Arduino board on a nearby table, with a laptop and tools in the background A hand-drawn diagram on a black background titled Milestone 1 - Physcomp shows two sketches. The first sketch illustrates a two gloves, which two people from the team wear and hide under the table to act like our solution so create an immersive user experience. The second sketch depicts a system where a servo motor pulls strings to move a thumb, connected to an Arduino board whihc connects to a finger to move it. This demonstrates our progress. A vinyl glove worn on a hand is covered with black marker drawings that outline the bones and joints of the fingers and palm. The glove is part of a low-fidelity prototype used to simulate the robotic hand. A team member wearing vinyl gloves with drawn-on joints demonstrates the hand gestures enabled by the prototype to the user, who observes attentively. The setting is a classroom workspace with laptops, props, and other prototype components on the table. A mechanical hand prototype is mounted on a wooden board, with its fingers controlled by strings connected to multiple servo motors. The setup is secured with duct tape, and various electronic components and wiring are visible, including an Arduino and breadboards. A person is manually adjusting one of the 3D-printed fingers, testing movement or alignment. Top-down view of a robotic hand prototype mounted on a wooden base. The fingers are connected to clear strings that lead to a newly designed servo bed. A hand is adjusting one of the servo motors, which control the finger movements in response to rotations. A person interacts with two robotic hand prototypes placed on a table, each connected to multiple servo motors and tangled wiring. The hands are mounted on black bases and linked to a breadboard and microcontroller setup. One hand appears to mirror or respond to the other, showing the gesture-mimicking system in development. A laptop sits nearby, used for programming or control. A complex breadboard configuration is shown on a table, connected to two robotic hands via numerous colored jumper wires. The setup includes a microcontroller and multiple servo motors, with duct tape and tools scattered around A robotic hand lies on a table with colored markers on its fingertips, while a smartphone and red marker pen sit nearby. A screen overlay shows finger positions being tracked in real-time using gesture recognition software, with lines and labels indicating finger joints and movements. A portion of code is visible on the side A robotic hand prototype rests on a table, with strings connected to servo motors that control finger movement. A hand is seen bending a flex sensor next to a breadboard, and the robotic hand mimics the bending motion in real-time, indicating successful gesture replication. Wires connect the system components, showcasing a functional flex-sensor-based control setup.

Physical Form
Our project features 2 identical mechanical right hands and forearms, 3D printed to mirror the movement of one hand to the other and vice versa. Each finger joint is bolted with 3mm by an assortment of 25, 20 and 15mm length screws and rotates a natural angle mimicking human hand movement. One hand mimics the other through a marionette system, as the finger changes position, the flex sensor works, triggering servo motors to turn, and the finger is able to curl and uncurl accordingly. The wrist rotation and flexion work based off 2 servo motor joints creating a 180 degree turning angle for the forearm in both axes. The base of our project can clamp to the side of the table, similar to a table mount in order to position the hand in a convenient place for users to access.

Technology Hardware

  • 3D printed plastic fingers, palm, wrist and servo bed. Basis - of the project.
  • Bolts, nuts, and screws. Connectors for the hand, fingers, and joints.
  • 10 micro servo motors. 180 degree turning angle. 9g torque.
  • 4 servo motors. 180 degree turning angle.
  • 2 arduino megas,
  • 2x 12V external power supply for the servo motor power.
  • Wiring to connect the entire project from arduino to boards to all functionality hardware.
  • Flex sensors 2.2" to detect flex angle changes in the fingers and reflect it to the receiving hand.
  • RP-C18.3-LT Thin Film Pressure sensors to detect taps, slaps, and give impact feedback.
  • Rgb Leds.

Software
Our project was coded with Arduino IDE. When the users interact with our project, a list of functions can be completed and some restrictions are placed. Users can only move hand when led is green, and will need to wait to receive gestures if the led if red (input output system). When a user manually moves a finger or multiple fingers, the flex sensor reacts and the micro servo motor is programmed to move using the marionette joint system until the flex of the other hand is identical. Users can also make movements to the rotation and flexion of the wrist to show positive and negative feedback and the servos are programmed to reflect identical change. After a gesture is made, the servo motors are programmed to return back to a default position (hand palm relaxed fingers straight). The pressure sensor at the top of the palm triggers a search function when activated (pairing mode). The sensor in the palm will trigger a high 5 where all fingers are upright and wrist is rotated to a convenient position. Pressure sensor on the knuckle will reflect a fist bump when activated only when all 3 flex sensors are bent in a way to reflect a fist curled up. This minimises false triggering and sets preconditions for our hands. We also coded a website to measure success and give incentives for completing games and tasks with the hand.

Technical decisions
For the mirroring of hands, we initially planned to utilise the string tension to move the servo spools directly. This proved to be ineffective due to the inconsistent tension when curling or uncurling the fingers. Our updated approach featured flex sensors on each individual finger, which measures the bend through a changed resistance. This "flex" value is received by the flex sensor on the other hand, and the corresponding servo motor spins until the second flex sensor reaches the same flex value. Furthermore, we planned to use a rounded wrist for the servo bed but realised it was too small due to its curved shape. We then designed a rectangular servo bed with a wrist holder to keep a consistent tension in the forearm for finger movement. Aside from these decisions, our spools went through several iterations. The first few were either too short, didn’t tense the strings when needed or didn’t release the tension. The second last iteration worked until the tension broke the printed plastic. Our final iteration which sealed our design decision was a 4 hole curved spool that was able to thread strings in a way to effectively switch tension when needed.