Toyota uses virtual reality to train robots for domestic tasks

27 Nov Toyota uses virtual reality to train robots for domestic tasks

On October 3, 2019, the Toyota Research Institute presented a way to train domestic robots using virtual reality. The trainer can thus visualize live and in 3D what the machine perceives to better teach it about household tasks. A necessary method because the disparity of the households prevents the creation of a single set of instructions.

Between 2000 and 2050, the proportion of the world’s population over 60 years of age will double from about 11% to 22%, according to estimates by the World Health Organization (WHO). For this reason, the Toyota Research Institute (TRI) is developing domestic robots that can do (almost) everything inside a home to help the elderly who have become dependent. A presentation video published on October 3, 2019, shows that the Japanese manufacturer is testing learning supervision using virtual reality.

At the prototype stage

The Toyota Research Institute is careful to point out that these machines are, for the time being, only prototypes and not products intended for commercialization. “We select tasks not to demonstrate our product concepts but to advance algorithm development,” the research institute prides itself on. However, it already imagines future applications outside the home, particularly in factories for repetitive tasks or even for logistics robots.

Modify the learning process

This learning process allows robots to be taught arbitrary tasks with a variety of distinct objects instead of predefined tasks with specific tools. Thus the machine can weave a link between what they see and the actions taught to them. This change of method makes it possible to make the robot more flexible when faced with a situation that is unknown to it. As a result, if he is facing a similar object but in a different scene, he will know which actions performed anyway. 

Home disparity as a barrier

Technically, it is challenging to train machines to operate in a home because each house is made unique by the layout and size of its rooms, the different storage units, the arrangement of dishes, the composition of its floor, among many things. In short, it is almost impossible to create a single set of instructions that would suit all households. To overcome this difficulty, engineers have devised a learning model using virtual reality. The technical details of this device have been compiled and published on the Cornell University (New York) website.

The idea is quite simple. Thanks to virtual reality headsets, the trainers put themselves in the robot’s shoes. They can thus see live and in 3D what the machine perceives thanks to sensors. This process facilitates movement direction and instructions. Teachers can annotate a scene to clarify things, such as how to grasp the handle of a refrigerator to open it.