Robotics is making great strides in a variety of areas, including some rather unusual ones. Researchers at the Idiap Research Institute in Switzerland, the Chinese University of Hong Kong (CUHK) and Wuhan University (WHU) have developed a machine learning-based method to teach robots to fry like professional chefs. according to the report TechXplore published on Friday.
Smart robots that can cook food
“Our recent work is a collaborative effort between three laboratories: the Robot Learning and Interaction Group led by Dr. Sylvain Calinon at the Idiap Research Institute, the Collaborative and Versatile Robot Laboratory led by Prof. Fei Chen Kuk, and the laboratory led by Prof. Miao Li from WHU,” said Junjia Liu, one of the researchers who conducted the study. TechXplore.
“Our three laboratories have been studying and working together for about ten years. We are especially interested in building intelligent robots that can cook food for humans.”
A new study hopes to create a robotic chef, something that has been very difficult to achieve so far.
“Although domestic service robots have been greatly improved in recent years, creating a robot chef in a semi-structured kitchen environment remains a big challenge,” Liu said.
“Cooking and cooking are the two most important activities in the household, and a robot chef that can follow arbitrary recipes and cook automatically will be practical and bring a new interactive entertainment experience.”
To complete a task as complex as roasting, Liu and his team first had to train a two-handed coordination model known as a “structured transformer.” They did this using human demonstrations.
“This mechanism considers coordination as a sequence transformation problem between both hand movements and uses a combined transducer and GNN model to achieve this,” Liu explained.
“So in the online process, the left hand movement is adjusted according to the visual feedback, and the corresponding right hand movement is generated by the pre-trained structured transducer model based on the left hand movement.”
We cook both at home and in public
Liu now hopes that his new and improved model will one day allow the development of robots that can cook food both at home and in public places. It can also be used in the development of robots capable of performing other tasks that require the use of two hands. A good example is the already popular pizza making robot.
“Now we will be inputting information from higher dimensions to explore more humanoid movements in kitchen skills, such as visual and electromyographic signals,” Liu concluded.
“Estimation of semifluid content in this work was simplified as 2D image segmentation, and we only used relative displacement as the desired target. Thus, we also plan to offer a more complete framework that includes both the movements of the bimanual manipulators and the change in the state of the object.”
study results were published in the journal IEEE Letters on Robotics and Automation.
This letter describes an approach to cooking the well known Chinese culinary art Stir Fry on a bimanual robotic system. Frying requires a sequence of highly dynamic coordinated movements that are usually difficult for a chef to learn, let alone transfer to robots. In this letter, we define the canonical hot motion and then offer an unrelated framework to explore this manipulation of a deformable object using a human demonstration as an example. First, the double arms of the robot are divided into different roles (leader and follower) and trained using classical and neural network methods separately, then the bimanual task is transformed into a coordination task. To obtain general bimanual coordination, we secondly propose a graph-transformer model, a structured transformer, to capture the spatio-temporal relationship between two hand movements. Finally, by adding visual feedback about the deformation of the content, our structure can automatically adjust the movements to achieve the desired frying effect. We test the structure with a simulator and deploy it on a real two-handed Panda robotic system. The experimental results confirm that our structure can realize the bimanual movement of the robot during frying and has the potential to propagate to other deformable objects with bimanual coordination.