A Self-learning Robot That Can Fend for Itself in Simulated Environments

There have always been two ways to teach swimming. One involves carefully instructing students on proper kick, stroke, and breathing techniques long before they are invited to cross the water on their own. The other is simpler: toss them in and let them figure it out for themselves. Those who have survived the latter can attest to its efficacy.
 
It works on microrobots as well. Researchers from Santa Clara University, New Jersey Institute of Technology, and the University of Hong Kong, created a simple, neural network-equipped virtual robot, dropped it in a simulated fluid and let it fend for itself. With nothing but a few points as targets, it managed to teach itself how to move and steer and was able to achieve the goals researchers set out for it: spelling the word swim.
 
“We wanted to see how the swimmer would act all by itself,” said Alan Tsang, a professor of mechanical engineering at the University of Hong Kong and one of the authors of the paper “Gait Switching and Targeted Navigation of Microswimmers via Deep Reinforcement Learning,” which appeared in June in Communications Physics. “We wanted to remove all pre-proscribed motions and see how it gets from point a to point b, all by itself.”

More Like This: This Robot Learns by Watching

The “smart” robot is a rudimentary tinker toy made up of three beads and two rods. This makes the self-training manageable, in terms of computational power.
 
“One can design an artificial swimmer that can do a lot more things than moving from one point to the next, but then the training process will be much more involved,” said Yuan-nan Young, a professor of mathematical sciences at the New Jersey Institute of Technology and one of the authors of the paper. “If you want to train a robot to swim like Michael Phelps, the swimmer will have a lot of complicated structures to perfect its swimming stroke.”
 
Not only was the stripped-down tadpole able to acquire enough skills to achieve its calligraphic aims, but it was also able to use them in new situations. When the team plopped the already trained robot into a more complex flow—with a small vortex—the robot successfully navigated out of the rapids and to its goal. It also outperformed a robot that had been pre-programmed with swimming abilities.

You Might Also Like: Robots That Climb Through Rubble

 “The knowledge the artificial swimmer has from training in the absence of flow is sufficient for it to work in the presence of flow,” said Young.
 
Such an autonomous learner will need some ability to sense things to become useful in the real world. “If it senses light reflected from plastic, for example, maybe this kind of swimmer could help clean the ocean,” said Young. He is now working to incorporate some kind of sensing, as well as the necessary information processing, into the training regimen.
 
Another requirement for usefulness is making the leap from virtual to actual. One of the authors of the paper, On Shun Pak, a professor of mechanical engineering at Santa Clara University, is working on a prototype of a real-world smart microrobot, to put the self-learning algorithm to the test.

Continue the Topic: Robotic Self-starters

The research could have myriad applications—targeted drug delivery, precise microsurgery, as well as the removal of plastics and other pollutants from our waterways—but the larger point of the research is to demonstrate the value of integrating different disciplines.
 
“The marriage between machine learning and microswimmers opens up an alternative avenue toward designing the next generation of microrobots,” Pak said. “This could help resolve the outstanding challenges in realizing the Fantastic Voyage.”
 
Michael Abrams is a science and technology writer in Westfield, N.J.