From Touch to Intelligence: Our Journey Towards Physical AI

For decades, robotic perception has been dominated by vision. Cameras have enabled robots to recognise objects, navigate complex environments, and understand scenes with remarkable accuracy. Yet humans rarely rely on vision alone. Every time we pick up a fragile glass, tighten a screw, or instinctively adjust our grip before an object slips, our sense of touch quietly takes over.

Recent advances across the robotics community - including high-resolution tactile sensing technologies such as GelSight and DIGIT, together with growing interest from organisations like Google DeepMind and NVIDIA - highlight an important shift. Touch is increasingly being recognised as one of the missing pieces needed for truly intelligent robots.

At Touchlab, this belief has driven our recent work. What began as an effort to detect slip has evolved into something much larger: building the foundations for tactile intelligence that will enable the next generation of dexterous robots.

Understanding Touch Before Building AI

Every successful AI model begins with understanding the data. Our journey started by characterising the behaviour of our tactile sensors. Before designing neural networks or collecting large datasets, we wanted to understand exactly how the fingertip behaves under different contact conditions, how force propagates through the sensing surface, and what information the sensor truly captures.

To achieve this, we developed a dedicated calibration system that ensures every sensor is calibrated consistently and repeatably. This provided a reliable foundation for collecting high-quality tactile measurements while giving us deeper insight into the behaviour of the sensor itself.

As our understanding improved, we realised that collecting meaningful tactile data required more than manual experiments. We therefore designed and built a robotic testing platform capable of executing controlled interactions automatically. This robotic jig allowed us to reproduce a wide range of grasping and manipulation behaviours while recording rich tactile information under repeatable conditions. This marked the transition from understanding the sensor to understanding touch itself.

Discovering the Signature of Slip

• 

• 

• 

Among the many behaviours we observed, one stood out. Slip leaves behind a distinctive tactile signature. By analysing thousands of controlled interactions, we were able to isolate the specific patterns associated with slipping while filtering out unrelated sensor information. These observations revealed the precise tactile features that distinguish stable contact from the onset of slip.

Rather than adapting an existing vision network, we designed a neural network architecture specifically for tactile slip detection. Training this model on carefully curated tactile datasets resulted in highly accurate real-time slip detection while remaining efficient enough for deployment at the edge.

More importantly, the project reinforced one of the biggest lessons in modern AI:

The quality of the dataset often matters more than the complexity of the model.

Computer vision experienced a step change through datasets such as ImageNet, while robotics is now undergoing a similar transformation through large-scale datasets including Open X-Embodiment, BridgeData and LeRobot. These efforts have demonstrated that diverse, representative application data is fundamental to building robust robotic intelligence.

Our experience was no different. Rich tactile datasets collected during real manipulation tasks became the most valuable component of the entire project.

Beyond Slip: Understanding Physical Interaction

Although slip detection was our initial objective, the data revealed something much more exciting. The same tactile information that enables slip detection also contains information about many other aspects of physical interaction.

This opens opportunities for:

• Dexterous manipulation

• Adaptive grasp control

• Contact state estimation

• Hardness estimation

• Edge classification

• Material interaction understanding

• Surface property recognition

In other words, touch is not simply another sensor modality - it is a rich source of information about the physical world.

From Benchmarking to SlipBot

As our ambitions grew, so did the platform, the result was SlipBot.

SlipBot is a robotic platform that combines tactile fingertips with a robot arm capable of autonomously generating a wide range of manipulation behaviours. Linear motion, torsional motion, rolling contact, combined motions, varying contact forces, controlled slip timing and different contact transitions can all be reproduced automatically with high repeatability.

Rather than producing a single experimental dataset, SlipBot continuously generates diverse, high-quality tactile datasets covering a broad spectrum of manipulation scenarios. Even more importantly, SlipBot was designed to be hardware agnostic.

Different tactile sensors, robotic hands and grippers can all be integrated into the platform, allowing rich datasets to be collected across multiple robotic systems using the same benchmarking methodology. This flexibility enables scalable data generation while supporting the wider robotics community.

Measuring What Touch Really Means

As our datasets expanded, another challenge became increasingly apparent, if tactile intelligence is going to become a core component of robotics, the community needs reliable ways to compare sensors, algorithms and manipulation performance.

To address this, we developed a benchmarking framework capable of evaluating fingertip behaviour under a wide variety of controlled interactions. Rather than evaluating only one application, the framework provides consistent metrics that help characterise tactile sensing performance across different tasks and operating conditions.

This benchmarking effort extends beyond Touchlab. Through our collaboration with ARIA, the platform is evolving into a shared evaluation environment where different companies can assess their tactile sensors, robotic hands and grippers using common methodologies. We believe that open benchmarking will play an important role in accelerating innovation across the entire robotics ecosystem.

Why Intelligent Touch Matters

The slip detection model demonstrates something much bigger than detecting when an object begins to move, it shows that tactile sensors are evolving from passive measurement devices into intelligent sensing systems.

When capabilities such as slip detection, edge classification, contact estimation and material recognition are embedded directly into the fingertip through edge AI, robots no longer need to interpret raw sensor values alone - they receive meaningful physical understanding in real time. This significantly reduces latency, lowers computational requirements and enables faster, more reliable manipulation.

Touch becomes actionable intelligence.

The Road Towards Physical AI

The future of robotics will not be driven by vision alone. Today's Vision-Language-Action (VLA) models, such as RT-2, have demonstrated how combining vision, language and robot actions allows robots to generalise beyond individual tasks. The next evolution is already beginning to emerge: incorporating tactile perception into these foundation models through Vision-Language-Tactile-Action (VLTA) systems.

Touch provides information that cameras simply cannot capture - contact quality, force distribution, friction, compliance and slip. These physical signals allow robots to understand interactions rather than merely observe them.

As tactile intelligence becomes integrated with vision, language models and world models, robots will gain a far richer understanding of the physical world. This will unlock safer manipulation, more dexterous grasping and increasingly capable autonomous systems.

We are excited to contribute to this vision through our collaborations with leading research groups, including Google DeepMind, where tactile intelligence is becoming an increasingly important component of next-generation Physical AI.

Looking Ahead

Our journey began with a simple question: Can a robot feel when an object is slipping?

Answering that question led us far beyond a single machine learning model. It led us to build calibration systems, robotic automation platforms, benchmarking frameworks, specialised neural networks, rich tactile datasets and SlipBot - a scalable platform for generating tactile intelligence.

Most importantly, it reinforced a belief that we think will define the next decade of robotics:

• Vision tells a robot what it is looking at.

• Touch tells a robot what is actually happening.

As the robotics community moves towards Physical AI, touch will become as fundamental as vision. We are proud to be contributing to that future - one interaction, one dataset and one intelligent fingertip at a time.

Shape the Future of Physical AI With Us

True tactile intelligence requires a collective effort across the robotics ecosystem. Whether you are developing next-generation foundation models, building robotic hardware, or looking to integrate robust tactile sensing into your platform, we want to collaborate.

Get in Touch with the Touchlab Team

Further Reading

• Deng et al. ImageNet: A Large-Scale Hierarchical Image Database. CVPR, 2009.

• Yuan et al. GelSight: High-Resolution Robot Tactile Sensors for Estimating Geometry and Force. 2017.

• Lambeta et al. DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor for Robot Manipulation. 2020.

• Brohan et al. RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control. 2023.

• Open X-Embodiment Collaboration. Open X-Embodiment: Robotic Learning Datasets and RT-X Models. 2024.

• LeRobot Project. LeRobot: Open Datasets, Models and Tools for Robotics. 2024.