Dexterous robotic hand integrates thermal, inertia and force sensors

Robotic players have introduced increasingly sophisticated systems over the past few decades, but most existing robots are unable to manipulate objects with the same dexterity and sensing capabilities as humans. This has a negative impact on performance on a variety of real-world tasks, from natural disasters, assembly and assembly execution, or housework after maintenance tasks to settlement of tile rubs, especially steel frames and casting environments.

Researchers at the University of Southern California recently developed motifs (multimodal observations with thermal, inertia, force sensors, and force sensors), a new robotic hand that can improve the object manipulation capabilities of humanoid robots. The innovations presented in a paper submitted to Arxiv Preprint Server feature a combination of sensing devices such as tactile sensors, depth sensors, thermal cameras, inertia measurement unit (IMU) sensors, and visual sensors.

“Our paper emerged from the need to advance robotic manipulation beyond traditional visual and tactile sensing,” Daniel Seita, hanyang Zhou, Wenhao Liu and Haozhe Lou told Tech Xplore. “Current hands of multi-tutoring robots often lack the integrated sensing capabilities needed for complex tasks that include thermal awareness and responsive contact feedback.”

The main purpose of recent research by Seita and his students was to develop new versatility and multimodal robotic hands inspired by the human hand. By utilizing more comprehensive sensor inputs, the hands they create can perform more subtle object interactions in a variety of environments, from home to factory.

“The first and most intuitive idea is to explore how to properly design a system that is more closely aligned with the daily perceptual experiences of humans. We were inspired by human behavioral activities,” explained Zhou and Li.

“We previously used the dexterous hands of some commercial robots. We noticed that many of our existing hardware designs focus on mechanical transmission structures, and the products we can purchase directly did not meet the sensing requirements of many research ideas.

“The motif hand is an advanced robotic hand that integrates several sensors: temperature, depth, RGB (visual), inertia (IMU), and tactile (a single dexterous platform),” the researchers explained. “It extends the widely used Leap Hand Design and offers enhanced sensing capabilities. Its key benefits include comprehensive environmental awareness, the ability to safely handle hot objects, and differentiate physical properties from similarly-looking objects.”

To assess the potential of motif hand, the researchers conducted two different experiments in a laboratory setting. The first experiment assessed the hand’s ability to grab objects while considering temperature and was designed to avoid touching particularly hot areas of the surface.

In the second experiment, we needed to manipulate objects with the same shape and different weights. To classify the mass of objects, the hands performed simple fingertip flick motions while processing them.

“The most notable achievement of our research is to integrate a variety of sensors into a single, dexterous robotic hand and significantly enhance its operating capabilities,” Lu said. “Our findings show that heat detection and 3D reconstruction effectively guide a safe grasp, and inertial sensing allows for accurate mass classification of objects.”

In the first test, the robotics introduced by researchers in this team worked very well, accurately predicting mass while still grasping the various objects safely. In the future, the motif hands could be further improved and tested in a wider range of experiments to verify the possibilities of a particular application.

Seita and his students believe that working on manual tasks in home environments, professional kitchens and industrial environments is particularly promising. The hands can safely handle hot objects and operate them accurately, thus surpassing other existing systems when cooking hot dishes, but in the case of welding or tightening, these tasks require accurate estimation of force and adaptation of the operational strategy based on the object’s temperature.

“In the future, we plan to further strengthen the motif hand by integrating additional high-resolution fingertip sensors (e.g. 360 digits) to provide even finer tactile feedback,” added Liu. “We also aim to refine our multimodal sensing algorithms and explore additional applications in complex real-world settings that require sophisticated manipulation tasks and sophisticated thermal and force-based interactions.”

“In this paper, we present the platform system. We presented this work at the ISER 2025 conference held in Santa Fe last week, and exchanged ideas with many great researchers. We really hope that thinking about more sensory modalities in the direction of research will inspire the community,” Zhou added.

©2025 Science X Network