Researchers teaching robots to use color when moving objects

By The Robot Report Staff | September 22, 2024

Research at Michigan State University is focused on teaching robots to use colors to perceive, visualize, and interpret interactions when manipulating objects. A force-interpreting optical system is being developed so robots can distinguish and manipulate soft and fragile objects – which will be particularly helpful for medical and other assistive robots.

“We’re working to integrate photoelastic gel into a stress-interpreting optical system for vision-based tactile gel-robots,” said Shaoting Lin, assistant professor of mechanical engineering. “Creating this perception for a robot will help it execute ultra-gentle manipulation of soft and fragile objects.”

Lin is the principal investigator in a three-year, $366,000 National Science Foundation collaboration. He is working with Civil Engineering Assistant Professor Wei Li at Stony Brook University and Industrial Engineering Assistant Professor Yu She at Purdue University.

“Specifically, this project will leverage the molecular design of fatigue-resistant photoelastic gels, the mechanical design of a stress-interpreting photometry system, and the algorithm design of physics-informed machine learning,” he explained.

Researchers have been designing robots with “soft” hands and computer vision systems for years. The technology helps reduce manual labor costs and improves efficiencies in a variety of medical and industrial applications, ranging from surgical robots to apple harvesting.

Lin said teaching assistive robots to use colors will advance their use in everything from collecting fragile jellies for marine studies to better domestic skills in assistive robots for serving food.

“Our long-term goal is to fill the fundamental gap that currently exists between robotic tactile perception and human haptic sensing,” Lin added. “It’s another step in preparing the next-generation of robots.”

Lin earned his Ph.D. degree (2019) at MIT and got his M.S. degree (2013) and B.S. degree (2010) at Tsinghua University. 

Editor’s Note: This are was republished from Michigan State University’s College of Engineering.