Exploring the Limits of Tactile Tele-Operation: Lessons from Our Recent Robotics Experiments

Tele-operation systems have traditionally relied on visual feedback, leaving operators to infer contact forces, slip, and surface properties from video alone. For manipulation tasks, this gap often results in conservative movements, dropped objects, or unintended collisions.

Recent advances in tactile sensing and wearable haptics offer a path toward richer, more intuitive tele-operation. In this post, we describe a series of tactile tele-robotics experiments in which a robot equipped with tactile sensors was paired with a haptic glove worn by a human operator. Our goal was to explore how touch feedback influences task performance, operator confidence, and overall system usability when interacting with a remote environment.

The Setup

Tactile information plays a central role in human manipulation, enabling fine force regulation, slip detection, and material discrimination. Translating this rich sensory channel into tele-operation, however, presents several challenges. Tactile sensors generate high-dimensional, high-frequency data, while haptic devices are often limited in bandwidth, resolution, or actuation strength.

Our motivation was to better understand which standard medical and day-to-day tasks could be performed using a single arm tele-operation system aided by tactile sensors.  Rather than attempting full tactile replication, we focused on conveying task-relevant cues (such as contact onset and force magnitude) through a HaptX G1 tactile glove.

The hardware consists of:

• Robot: A Universal Robots UR10e arm.

• Hand: A Shadow Dexterous hand equipped with Touchlab Fingertips. These consist of a 16-sensor array with a 1-to-1 mapping in placement and number with the actuators in the haptic glove.

• Control: Arm motion is controlled via an HTC Vive tracker mounted on the operator's wrist.

Evaluating Tactile Tele-operation through Dexterous Tasks

We designed a set of manipulation tasks that explicitly require tactile awareness, including grasping day-to-day objects, inserting components with tight tolerances, and handling basic medical tools. Each task was performed with the operator in proximity to the arm to see the scene. Operators were instructed to prioritise task success and object safety rather than speed. The only performance metrics were a simple task success or failure, and with all successful tasks received a qualitative difficulty rating.

The following scenarios were explored:

• Opening and closing a water bottle

• The use of keys

• Stacking and removing plates and cups from a rack

• Attaching and removing a carabiner

• Applying duct tape to a box

• Using scissors and wire clippers

• Using a box cutter knife

• Using a drill

• Inserting and turning on a plug

• Attaching and removing crocodile clips

• Inserting and removing USB and ethernet cables

• Using a syringe

• Wrapping a bandage around an arm

• Inserting a urinary catheter into a prop

OBJECT	SUCCESS	QUALITATIVE DIFFICULTY SCORE (1-10)	PRIMARY CHALLENGE
Water Bottle	Yes	5	Lack of DOF at wrist
Keys	No	-	Lack of fingertip compliance
Stacking Plates	Yes	4	Lack of DOF at wrist
Carabiner	No	-	Lack of fingertip compliance
Duct Tape	Yes	8	Grasp stability
Scissors and Clippers	No	-	Fingertip size and lack of compliance
Box Cutter	No	-	Finger tracking and lack of compliance on phalanges
Drill	No	-	Insufficient index finger flexion
Plug and Switch	Yes	6	Grip strength
Crocodile Clips	No	-	Grip strength and lack of compliance
USB Cable	Yes	6	Grip strength
Ethernet Cable	No	-	Grip strength
Syringe	No	-	Lack of compliance on phalanges
Bandage	No	-	Lack of DOF at wrist
Urinary Catheter	Yes	4	Finger tracking

Tactile Precision in Action

To see the system’s potential for high-stakes precision, watch the video below. It shows the urinary catheter insertion performed through the tele-operation setup.

Despite the complexity of the medical tools, this task received a relatively low difficulty rating of 4. The video highlights how the operator is able to "feel" the tool’s interaction with the prop, allowing for a smooth and confident insertion that would be significantly more difficult—and riskier—using visual feedback alone.

Lessons Learned

Across all tasks, the addition of tactile feedback led to more deliberate and confident manipulation.  Operators were able to perceive tactile confirmation of interactions with the props, often responding to tactile cues before visual ones. This was most apparent when grasping an object where the fingers obscured the line of sight. The benefits were most pronounced when the operator's view was obscured, forcing them to rely heavily on the haptic sense.

Despite some successes, several challenges remain:

• Dimensionality: The sensor data received by the system is far richer than what can currently be passed to the operator, requiring aggressive dimensionality reduction that inevitably discards information.

• Wrist Mobility: The wrist joint of the Shadow hand was not used in these experiments, which made certain motions (like using a drill or box cutter) less intuitive.

• The Compliance Issue: Both the Shadow hand and the sensors lacked a degree of compliance (softness) which humans naturally rely on to anchor objects. The syringe was a perfect example; the normal grip a human would use resulted in "point contacts" that were unstable on the rigid robotic surface.

Key Takeaways

For future iterations, it is clear that the sensors—and ideally the robot hand itself—should incorporate some level of mechanical compliance to securely grip objects designed for human hands. Furthermore, increasing the Degrees of Freedom (DOF) at the wrist and expanding the sensorised area of the hand beyond just the fingertips will be critical for more accurate and intuitive dexterous interactions.

Join the Journey

Tactile tele-dexterity is an evolving field, and we are already planning our next round of tests—focusing on mechanical compliance and expanded sensor arrays.

Stay updated on our progress by signing up for our monthly newsletter. Every month, we share:

• Early access to experiment data and videos.

• Hardware reviews (sensors, arms, and haptic gloves).

• Deep dives into the software challenges of tele-robotics.

Subscribe to the Monthly Newsletter