ICRA 2022 Workshop on Releasing Robots into the Wild: Simulations, Benchmarks, and Deployment

Overview

Machine learning and learning-based control promise to lead robot capabilities and performance beyond what human designs can attain. However, the results from recent years of fast-paced progress in reinforcement learning and learning-based control have proven challenging to compare. While acknowledging that learning robots carry a plethora of fascinating open research questions, in this workshop, we specifically eye physics-based simulations and learning for decision making. We argue that two important roadblocks hamper the transfer of robot learning research into real-world applications: (i) the scarcity of sufficiently realistic simulation tools, tasks, and datasets to reliably compare algorithmic progress; and (ii) the lack of reliable and repeatable processes to transfer those simulation results to the real-world.

Discussion Themes | Speakers | Program | Organizers

Discussion Themes

Theme A: Realistic Simulation Tools, Tasks, and Datasets to Reliably Compare Algorithmic Progress

  • Many simulators exist, but yet none has really become a common benchmark. What is missing?
  • What constitutes a fair benchmark, how do we design for them, and how can we avoid overfitting to the specific benchmarks/datasets?
  • What benchmark tasks and metrics should we prioritize, do they have sufficient coverage over our desired robot performance, e.g. in terms of efficiency, safety and robustness?
  • How does/can simulation help benchmarking and robotics research, what are the desirable robot simulator properties, what is hampering their implementations and what are their limitations?

Theme B: Reliable and Repeatable Processes to Transfer Simulation Results to the Real World

  • The Turing test for robotics: what should be tested in an autonomous system to guarantee its functionality in unstructured environments?
  • How can we guarantee that the conclusions we draw in the simulation are also valid in the physical world? Shall we rely on a (possibly utopic) algorithm to perfectly transfer knowledge between domains or can we do something better?
  • What are the biggest issues when transferring an algorithm from simulation to the real world, how these issues can be addressed, and can the solutions to these issues be made reliable and scalable, leveraging standard engineering practices, for real-life deployment?
  • We can never exactly simulate the real world, thus, we cannot expect perfect transfer. However, can we define a bound/set an expectation of how well we theoretically expect a robot, trained in simulation, to perform in the real world?

Invited Speakers

Theme A: Realistic Simulation Tools, Tasks, and Datasets to Reliably Compare Algorithmic Progress

  • Joelle Pineau from McGill and Facebook Research (confirmed). Tentative talk title: “Building Reproducible, Reusable, and Robust Deep Learning Systems”
  • Joshua Achiam from OpenAI, creator of safety-gym (confirmed). Tentative talk title: “Challenges in Benchmarking Safety”
  • Erwin Coumans from Google Brain and the PyBullet development team (confirmed). Tentative talk title: “Simulation and Sim-to-Real for Quadruped Robot Locomotion”
  • Liila Torabi from Nvidia and the Isaac team (confirmed). Tentative talk title: “NVIDIA Isaac Sim: A Platform for Developing and Training Smarter Robots”
  • Karime Pereida from Ocado Technology (confirmed). Tentative talk title: “From Simulation to Production: Reinforcement Learning in Industry”

Theme B: Reliable and Repeatable Processes to Transfer Simulation Results to the Real World

  • Jens Kober from TU Delft (confirmed). Tentative talk title: “Interactions: A Blessing or a Curse?”
  • Ingmar Posner from Oxford and Oxbotica (confirmed). Tentative talk title: “Learning to Simulate: Generating Data for Perception and Action”
  • Raquel Urtasun from University of Toronto and Waabi (confirmed). Talk title TBD
  • David Hsu from the National University of Singapore (confirmed). Tentative talk title: “Closing the Planning-Learning  Loop: Autonomous Driving, Object Manipulation,…”
  • Animesh Garg from the University of Toronto and Nvidia (confirmed). Tentative talk title: “Paving the Path to Robot Autonomy with Simulation

Program

Below is a tentative schedule of our workshop. Times are in EST.

Morning

09:50 – 10:00: Opening Remarks
10:00 – 10:10: Theme A Introduction
10:10 – 11:00: Theme A Invited Speaker Session
11:10 – 11:20: Coffee Break
11:20 – 12:05: Theme A Panel Discussion
12:05 – 12:25: Theme A Lightning Talks
12:25 – 12:45: Theme A Poster Session
12:45 – 13:30: Lunch Break

Afternoon

13:30 – 13:40: Theme B Introduction
13:40 – 14:30: Theme B Invited Speaker Session
14:30 – 14:40: Coffee Break
14:40 – 15:25: Theme B Panel Discussion
15:25 – 15:45: Theme B Lightning Talks
15:45 – 16:10: Theme B Poster Session
16:10 – 16:20: Concluding Remarks
16:20 – 17:00: Social Event

Organizers

Angela P. Schoellig, Associate Professor, Institute for Aerospace Studies, University of Toronto
Davide Scaramuzza, Professor, Department of Informatics, University of Zurich
Adam Hall, Ph.D. Student, Institute for Aerospace Studies, University of Toronto
Zhaocong Yuan, MASc Student, Institute for Aerospace Studies, University of Toronto
Jacopo Panerati, Postdoctoral Fellow, Institute for Aerospace Studies, University of Toronto
SiQi Zhou, Ph.D. Candidate, Institute for Aerospace Studies, University of Toronto
Lukas Brunke, Ph.D. Student, Institute for Aerospace Studies, University of Toronto
Melissa Greeff, Ph.D. Candidate, Institute for Aerospace Studies, University of Toronto

University of Toronto Institute for Aerospace Studies