Pieter Abbeel

ACM named Pieter Abbeel the recipient of the 2021 ACM Prize in Computing for contributions to robot learning, including learning from demonstrations and deep reinforcement learning for robotic control. Abbeel pioneered teaching robots to learn from human demonstrations (“apprenticeship learning”) and through their own trial and error (“reinforcement learning”), which have formed the foundation for the next generation of robotics. Abbeel is a Professor at the University of California, Berkeley and the Co-Founder, President and Chief Scientist at Covariant, an AI robotics company.

Early in his career, Abbeel developed new apprenticeship learning techniques to significantly improve robotic manipulation. As the field matured, researchers were able to program robots to perceive and manipulate rigid objects such as wooden blocks or spoons. However, programming robots to manipulate deformable objects, such as cloth, proved difficult because the way soft materials move when touched is unpredictable. Abbeel introduced new methods to enhance robot visual perception, physics-based tracking, control, and learning from demonstration. By combining these new methods, Abbeel developed a robot that was able to fold clothes such as towels and shirts ─ an improvement over existing technology that was considered an important milestone at the time.

Abbeel’s contributions also include developing robots that can perform surgical suturing, detect objects, and plan their trajectories in uncertain situations. More recently, he has pioneered “few-shot imitation learning,” where a robot is able to learn to perform a task from just one demonstration after having been pre-trained with a large set of demonstrations on related tasks.

Another especially promising area where Abbeel has made important contributions is in deep reinforcement learning for robotics. Reinforcement learning is an area of machine learning where an agent (e.g., a computer program) seeks to progress towards a reward (e.g., winning a game). While early reinforcement learning programs were effective, they could only perform simple tasks. The innovation of combining reinforcement learning with deep neural networks ushered in the new field of deep reinforcement learning, which can solve far more complex problems than computer programs developed with reinforcement learning alone.

Abbeel’s key breakthrough contribution in this area was developing a deep reinforcement learning method called Trust Region Policy Optimization. This method stabilizes the reinforcement learning process, enabling robots to learn a range of simulated control skills. By sharing his results, posting video tutorials, and releasing open-source code from his lab, Abbeel helped build a community of researchers that has since pushed deep learning for robotics even further ─ with robots performing ever more complicated tasks.

Abbeel has also made several other pioneering contributions including: generalized advantage estimation, which enabled the first 3D robot locomotion learning; soft-actor critic, which is one of the most popular deep reinforcement learning algorithms to-date; domain randomization, which showcases how learning across appropriately randomized simulators can generalize surprisingly well to the real world; and hindsight experience replay, which has been instrumental for deep reinforcement learning in sparse-reward/goal-oriented environments.

“Teaching robots to learn could spur major advances across many industries ─ from surgery and manufacturing to shipping and automated driving,” said ACM President Gabriele Kotsis. “Pieter Abbeel is a recognized leader among a new generation of researchers who are harnessing the latest machine learning techniques to revolutionize this field. Abbeel has made leapfrog research contributions, while also generously sharing his knowledge to build a community of colleagues working to take robots to an exciting new level of ability. His work exemplifies the intent of the ACM Prize in Computing to recognize outstanding work with ‘depth, impact, and broad implications.’”

“Infosys is proud of our longstanding collaboration with ACM, and we are honored to recognize Pieter Abbeel for the 2021 ACM Prize in Computing,” said Salil Parekh, Chief Executive Officer, Infosys. “The robotics field is poised for even greater advances, as innovative new ways are emerging to combine robotics with AI, and we believe researchers like Abbeel will be instrumental in creating the next great advances in this field.”

Abbeel will be formally presented with the ACM Prize in Computing at the annual ACM Awards Banquet, which will be held this year on Saturday, June 11 at the Palace Hotel in San Francisco.