## ICRA 2019 Home » ICRA 2019

We had a strong presence at the IEEE International Conference on Robotics and Automation (ICRA) in 2019, with five accepted papers and five accepted workshop items (see photos and schedule below).

### Papers

There’s no place like home: visual teach and repeat for emergency return of multirotor UAVs during GPS failure
M. Warren, M. Greeff, B. Patel, J. Collier, A. P. Schoellig, and T. D. Barfoot
IEEE Robotics and Automation Letters, vol. 4, iss. 1, p. 161–168, 2019.

Redundant navigation systems are critical for safe operation of UAVs in high-risk environments. Since most commercial UAVs almost wholly rely on GPS, jamming, interference and multi-pathing are real concerns that usually limit their operations to low-risk environments and VLOS. This paper presents a vision-based route-following system for the autonomous, safe return of UAVs under primary navigation failure such as GPS jamming. Using a Visual Teach and Repeat framework to build a visual map of the environment during an outbound flight, we show the autonomous return of the UAV by visually localising the live view to this map when a simulated GPS failure occurs, controlling the vehicle to follow the safe outbound path back to the launch point. Using gimbal-stabilised stereo vision alone, without reliance on external infrastructure or inertial sensing, Visual Odometry and localisation are achieved at altitudes of 5-25 m and flight speeds up to 55 km/h. We examine the performance of the visual localisation algorithm under a variety of conditions and also demonstrate closed-loop autonomy along a complicated 450 m path.

@article{warren-ral19,
title = {There's No Place Like Home: Visual Teach and Repeat for Emergency Return of Multirotor {UAV}s During {GPS} Failure},
author = {Michael Warren and Melissa Greeff and Bhavit Patel and Jack Collier and Angela P. Schoellig and Timothy D. Barfoot},
journal = {{IEEE Robotics and Automation Letters}},
year = {2019},
volume = {4},
number = {1},
pages = {161--168},
doi = {10.1109/LRA.2018.2883408},
urlvideo = {https://youtu.be/oJaQ4ZbvsFw},
abstract = {Redundant navigation systems are critical for safe operation of UAVs in high-risk environments. Since most commercial UAVs almost wholly rely on GPS, jamming, interference and multi-pathing are real concerns that usually limit their operations to low-risk environments and VLOS. This paper presents a vision-based route-following system for the autonomous, safe return of UAVs under primary navigation failure such as GPS jamming. Using a Visual Teach and Repeat framework to build a visual map of the environment during an outbound flight, we show the autonomous return of the UAV by visually localising the live view to this map when a simulated GPS failure occurs, controlling the vehicle to follow the safe outbound path back to the launch point. Using gimbal-stabilised stereo vision alone, without reliance on external infrastructure or inertial sensing, Visual Odometry and localisation are achieved at altitudes of 5-25 m and flight speeds up to 55 km/h. We examine the performance of the visual localisation algorithm under a variety of conditions and also demonstrate closed-loop autonomy along a complicated 450 m path.}
}

Trajectory generation for multiagent point-to-point transitions via distributed model predictive control
C. E. Luis and A. P. Schoellig
IEEE Robotics and Automation Letters, vol. 4, iss. 2, p. 357–382, 2019.

This paper introduces a novel algorithm for multiagent offline trajectory generation based on distributed model predictive control (DMPC). By predicting future states and sharing this information with their neighbours, the agents are able to detect and avoid collisions while moving towards their goals. The proposed algorithm computes transition trajectories for dozens of vehicles in a few seconds. It reduces the computation time by more than 85\% compared to previous optimization approaches based on sequential convex programming (SCP), with only causing a small impact on the optimality of the plans. We replaced the previous compatibility constraints in DMPC, which limit the motion of the agents in order to avoid collisions, by relaxing the collision constraints and enforcing them only when required. The approach was validated both through extensive simulations for a wide range of randomly generated transitions and with teams of up to 25 quadrotors flying in confined indoor spaces.

@article{luis-ral19,
title = {Trajectory Generation for Multiagent Point-To-Point Transitions via Distributed Model Predictive Control},
author = {Carlos E. Luis and Angela P. Schoellig},
journal = {{IEEE Robotics and Automation Letters}},
year = {2019},
volume = {4},
number = {2},
pages = {357--382},
urlvideo = {https://youtu.be/ZN2e7h-kkpw},
abstract = {This paper introduces a novel algorithm for multiagent offline trajectory generation based on distributed model predictive control (DMPC). By predicting future states and sharing this information with their neighbours, the agents are able to detect and avoid collisions while moving towards their goals. The proposed algorithm computes transition trajectories for dozens of vehicles in a few seconds. It reduces the computation time by more than 85\% compared to previous optimization approaches based on sequential convex programming (SCP), with only causing a small impact on the optimality of the plans. We replaced the previous compatibility constraints in DMPC, which limit the motion of the agents in order to avoid collisions, by relaxing the collision constraints and enforcing them only when required. The approach was validated both through extensive simulations for a wide range of randomly generated transitions and with teams of up to 25 quadrotors flying in confined indoor spaces.}
}

Learn fast, forget slow: safe predictive control for systems with locally linear actuator dynamics performing repetitive tasks
C. D. McKinnon and A. P. Schoellig
IEEE Robotics and Automation Letters, vol. 4, iss. 2, p. 2180–2187, 2019.

We present a control method for improved repetitive path following for a ground vehicle that is geared towards long-term operation where the operating conditions can change over time and are initially unknown. We use weighted Bayesian Linear Regression to model the unknown actuator dynamics, and show how this simple model is more accurate in both its estimate of the mean behaviour and model uncertainty than Gaussian Process Regression and generalizes to novel operating conditions with little or no tuning. In addition, it allows us to use fast adaptation and long-term learning in one, unified framework, to adapt quickly to new operating conditions and learn repetitive model errors over time. This comes with the added benefit of lower computational cost, longer look-ahead, and easier optimization when the model is used in a robust, Model Predictive controller (MPC). In order to fully capitalize on the long prediction horizons that are possible with this new approach, we use Tube MPC to reduce predicted uncertainty growth. We demonstrate the effectiveness of our approach in experiment on a 900 kg ground robot showing results over 2.7 km of driving with both physical and artificial changes to the robot’s dynamics. All of our experiments are conducted using a stereo camera for localization.

@article{mckinnon-ral19,
title={Learn Fast, Forget Slow: Safe Predictive Control for Systems with Locally Linear Actuator Dynamics Performing Repetitive Tasks},
author={Christopher D. McKinnon and Angela P. Schoellig},
journal = {{IEEE Robotics and Automation Letters}},
year = {2019},
volume = {4},
number = {2},
pages = {2180--2187},
urlvideo={https://youtu.be/fLNMtYabuU4},
abstract = {We present a control method for improved repetitive path following for a ground vehicle that is geared towards long-term operation where the operating conditions can change over time and are initially unknown. We use weighted Bayesian Linear Regression to model the unknown actuator dynamics, and show how this simple model is more accurate in both its estimate of the mean behaviour and model uncertainty than Gaussian Process Regression and generalizes to novel operating conditions with little or no tuning. In addition, it allows us to use fast adaptation and long-term learning in one, unified framework, to adapt quickly to new operating conditions and learn repetitive model errors over time. This comes with the added benefit of lower computational cost, longer look-ahead, and easier optimization when the model is used in a robust, Model Predictive controller (MPC). In order to fully capitalize on the long prediction horizons that are possible with this new approach, we use Tube MPC to reduce predicted uncertainty growth. We demonstrate the effectiveness of our approach in experiment on a 900 kg ground robot showing results over 2.7 km of driving with both physical and artificial changes to the robot's dynamics. All of our experiments are conducted using a stereo camera for localization.}
}

Building a winning self-driving car in six months
K. Burnett, A. Schimpe, S. Samavi, M. Gridseth, C. W. Liu, Q. Li, Z. Kroeze, and A. P. Schoellig
in Proc. of the IEEE International Conference on Robotics and Automation (ICRA), 2019, p. 9583–9589.

The SAE AutoDrive Challenge is a three-year competition to develop a Level 4 autonomous vehicle by 2020. The first set of challenges were held in April of 2018 in Yuma, Arizona. Our team (aUToronto/Zeus) placed first. In this paper, we describe our complete system architecture and specialized algorithms that enabled us to win. We show that it is possible to develop a vehicle with basic autonomy features in just six months relying on simple, robust algorithms. We do not make use of a prior map. Instead, we have developed a multi-sensor visual localization solution. All of our algorithms run in real-time using CPUs only. We also highlight the closed-loop performance of our system in detail in several experiments.

@INPROCEEDINGS{burnett-icra19,
author = {Keenan Burnett and Andreas Schimpe and Sepehr Samavi and Mona Gridseth and Chengzhi Winston Liu and Qiyang Li and Zachary Kroeze and Angela P. Schoellig},
title = {Building a Winning Self-Driving Car in Six Months},
booktitle = {{Proc. of the IEEE International Conference on Robotics and Automation (ICRA)}},
year = {2019},
pages = {9583--9589},
urlvideo = {http://tiny.cc/zeus-y1},
abstract = {The SAE AutoDrive Challenge is a three-year competition to develop a Level 4 autonomous vehicle by 2020. The first set of challenges were held in April of 2018 in Yuma, Arizona. Our team (aUToronto/Zeus) placed first. In this paper, we describe our complete system architecture and specialized algorithms that enabled us to win. We show that it is possible to develop a vehicle with basic autonomy features in just six months relying on simple, robust algorithms. We do not make use of a prior map. Instead, we have developed a multi-sensor visual localization solution. All of our algorithms run in real-time using CPUs only. We also highlight the closed-loop performance of our system in detail in several experiments.},
}

Fast and in sync: periodic swarm patterns for quadrotors
X. Du, C. E. Luis, M. Vukosavljev, and A. P. Schoellig
in Proc. of the IEEE International Conference on Robotics and Automation (ICRA), 2019, p. 9143–9149.

This paper aims to design quadrotor swarm performances, where the swarm acts as an integrated, coordinated unit embodying moving and deforming objects. We divide the task of creating a choreography into three basic steps: designing swarm motion primitives, transitioning between those movements, and synchronizing the motion of the drones. The result is a flexible framework for designing choreographies comprised of a wide variety of motions. The motion primitives can be intuitively designed using few parameters, providing a rich library for choreography design. Moreover, we combine and adapt existing goal assignment and trajectory generation algorithms to maximize the smoothness of the transitions between motion primitives. Finally, we propose a correction algorithm to compensate for motion delays and synchronize the motion of the drones to a desired periodic motion pattern. The proposed methodology was validated experimentally by generating and executing choreographies on a swarm of 25 quadrotors.

@INPROCEEDINGS{du-icra19,
author = {Xintong Du and Carlos E. Luis and Marijan Vukosavljev and Angela P. Schoellig},
title = {Fast and in sync: periodic swarm patterns for quadrotors},
booktitle = {{Proc. of the IEEE International Conference on Robotics and Automation (ICRA)}},
year = {2019},
pages={9143--9149},
abstract = {This paper aims to design quadrotor swarm performances, where the swarm acts as an integrated, coordinated unit embodying moving and deforming objects. We divide the task of creating a choreography into three basic steps: designing swarm motion primitives, transitioning between those movements, and synchronizing the motion of the drones. The result is a flexible framework for designing choreographies comprised of a wide variety of motions. The motion primitives can be intuitively designed using few parameters, providing a rich library for choreography design. Moreover, we combine and adapt existing goal assignment and trajectory generation algorithms to maximize the smoothness of the transitions between motion primitives. Finally, we propose a correction algorithm to compensate for motion delays and synchronize the motion of the drones to a desired periodic motion pattern. The proposed methodology was validated experimentally by generating and executing choreographies on a swarm of 25 quadrotors.},
}

### Workshop Items

Robust adaptive model predictive control for high-accuracy trajectory tracking in changing conditions
K. Pereida and A. P. Schoellig
Short Paper and Presentation, in Proc. of the Algorithms and Architectures for Learning in-the-Loop Systems in Autonomous Flight Workshop at IEEE International Conference on Robotics and Automation (ICRA), 2019.

Robots are being deployed in unknown and dynamic environments where they are required to handle disturbances, unmodeled dynamics, and parametric uncertainties. Sophisticated control strategies can guarantee high performance in these changing environments. In this work, we propose a novel robust adaptive model predictive controller that combines robust model predictive control (MPC) with an underlying $\mathcal{L}_1$ adaptive controller to improve trajectory tracking of a system subject to unknown and changing disturbances. The $\mathcal{L}_1$ adaptive controller forces the system to behave close to a specified linear reference model. The controlled system may still deviate from the reference model, but this deviation is shown to be upper bounded. An outer-loop robust MPC uses this upper bound, the linear reference model and system constraints to calculate the optimal reference input that minimizes the given cost function. The proposed robust adaptive MPC is able to achieve high-accuracy trajectory tracking even in the presence of unknown disturbances. We show preliminary experimental results of an adaptive MPC on a quadrotor. The adaptive MPC has a lower trajectory tracking error compared to a predictive, non-adaptive approach, even when wind disturbances are applied.

@MISC{pereida-icra19a,
author = {Karime Pereida and Angela P. Schoellig},
title = {Robust Adaptive Model Predictive Control for High-Accuracy Trajectory Tracking in Changing Conditions},
year = {2019},
howpublished = {Short Paper and Presentation, in Proc. of the Algorithms and Architectures for Learning in-the-Loop Systems in Autonomous Flight Workshop at IEEE International Conference on Robotics and Automation (ICRA)},
urlvideo = {https://youtu.be/xuyLst5mkEE},
abstract = {Robots are being deployed in unknown and dynamic environments where they are required to handle disturbances, unmodeled dynamics, and parametric uncertainties. Sophisticated control strategies can guarantee high performance in these changing environments. In this work, we propose a novel robust adaptive model predictive controller that combines robust model predictive control (MPC) with an underlying $\mathcal{L}_1$ adaptive controller to improve trajectory tracking of a system subject to unknown and changing disturbances. The $\mathcal{L}_1$ adaptive controller forces the system to behave close to a specified linear reference model. The controlled system may still deviate from the reference model, but this deviation is shown to be upper bounded. An outer-loop robust MPC uses this upper bound, the linear reference model and system constraints to calculate the optimal reference input that minimizes the given cost function. The proposed robust adaptive MPC is able to achieve high-accuracy trajectory tracking even in the presence of unknown disturbances. We show preliminary experimental results of an adaptive MPC on a quadrotor. The adaptive MPC has a lower trajectory tracking error compared to a predictive, non-adaptive approach, even when wind disturbances are applied.},
}

Diversity in robotics: from diverse teams to diverse impact
K. Pereida and M. Greeff
Short Paper and Presentation, in Proc. of the Debates on the Future of Robotics Research Workshop at the IEEE International Conference on Robotics and Automation (ICRA), 2019.

Roboticists develop technologies that are used by people worldwide, consequently impacting many aspects of human life – from healthcare and law enforcement to autonomous transportation. The development of these technologies involves design and innovation – both of which rely on personal choice and experience. Hence, personal biases, whether intentionally or unintentionally, tend to be embedded in the final product designs. Homogeneous teams of designers and engineers are more likely to develop products that overlook the needs of a given part of the population – even missing gaps for potential technological innovation. In this talk we emphasize some of the negative impacts a lack of diversity has on robotic innovation by highlighting examples of embedded biases within certain technologies and providing some evidence that this is linked to a lack of diverse teams. If our aim as a community is to increase research capacity, creativity, and broaden the impact of robotics, making it a more diverse field must be a goal.

@MISC{pereida-icra19b,
author = {Karime Pereida and Melissa Greeff},
title = {Diversity in Robotics: From Diverse Teams to Diverse Impact},
year = {2019},
howpublished = {Short Paper and Presentation, in Proc. of the Debates on the Future of Robotics Research Workshop at the IEEE International Conference on Robotics and Automation (ICRA)},
abstract = {Roboticists develop technologies that are used by people worldwide, consequently impacting many aspects of human life - from healthcare and law enforcement to autonomous transportation. The development of these technologies involves design and innovation - both of which rely on personal choice and experience. Hence, personal biases, whether intentionally or unintentionally, tend to be embedded in the final product designs. Homogeneous teams of designers and engineers are more likely to develop products that overlook the needs of a given part of the population - even missing gaps for potential technological innovation. In this talk we emphasize some of the negative impacts a lack of diversity has on robotic innovation by highlighting examples of embedded biases within certain technologies and providing some evidence that this is linked to a lack of diverse teams. If our aim as a community is to increase research capacity, creativity, and broaden the impact of robotics, making it a more diverse field must be a goal.},
}

Data-efficient multi-robot, multi-task transfer learning for trajectory tracking
K. Pereida, M. K. Helwa, and A. P. Schoellig
Abstract and Poster, in Proc. of the Resilient Robot Teams Workshop at the IEEE International Conference on Robotics and Automation (ICRA), 2019.

Learning can significantly improve the performance of robots in uncertain and changing environments; however, typical learning approaches need to start a new learning process for each new task or robot as transferring knowledge is cumbersome or not possible. In this work, we introduce a multi-robot, multi-task transfer learning framework that allows a system to complete a task by learning from a few demonstrations of another task executed on a different system. We focus on the trajectory tracking problem where each trajectory represents a different task. The proposed learning control architecture has two stages: (i) \emph{multi-robot} transfer learning framework that combines $\mathcal{L}_1$ adaptive control and iterative learning control, where the key idea is that the adaptive controller forces dynamically different systems to behave as a specified reference model; and (ii) a \emph{multi-task} transfer learning framework that uses theoretical control results (e.g., the concept of vector relative degree) to learn a map from desired trajectories to the inputs that make the system track these trajectories with high accuracy. This map is used to calculate the inputs for a new, unseen trajectory. We conduct experiments on two different quadrotor platforms and six different trajectories where we show that using information from tracking a single trajectory learned by one quadrotor reduces, on average, the first-iteration tracking error on another quadrotor by 74\%.

@MISC{pereida-icra19c,
author = {Karime Pereida and Mohamed K. Helwa and Angela P. Schoellig},
title = {Data-Efficient Multi-Robot, Multi-Task Transfer Learning for Trajectory Tracking},
year = {2019},
howpublished = {Abstract and Poster, in Proc. of the Resilient Robot Teams Workshop at the IEEE International Conference on Robotics and Automation (ICRA)},
abstract = {Learning can significantly improve the performance of robots in uncertain and changing environments; however, typical learning approaches need to start a new learning process for each new task or robot as transferring knowledge is cumbersome or not possible. In this work, we introduce a multi-robot, multi-task transfer learning framework that allows a system to complete a task by learning from a few demonstrations of another task executed on a different system. We focus on the trajectory tracking problem where each trajectory represents a different task. The proposed learning control architecture has two stages: (i) \emph{multi-robot} transfer learning framework that combines $\mathcal{L}_1$ adaptive control and iterative learning control, where the key idea is that the adaptive controller forces dynamically different systems to behave as a specified reference model; and (ii) a \emph{multi-task} transfer learning framework that uses theoretical control results (e.g., the concept of vector relative degree) to learn a map from desired trajectories to the inputs that make the system track these trajectories with high accuracy. This map is used to calculate the inputs for a new, unseen trajectory. We conduct experiments on two different quadrotor platforms and six different trajectories where we show that using information from tracking a single trajectory learned by one quadrotor reduces, on average, the first-iteration tracking error on another quadrotor by 74\%.},
}

Towards scalable online trajectory generation for multi-robot systems
C. E. Luis, M. Vukosavljev, and A. P. Schoellig
Abstract and Poster, in Proc. of the Resilient Robot Teams Workshop at the IEEE International Conference on Robotics and Automation (ICRA), 2019.

We present a distributed model predictive control (DMPC) algorithm to generate trajectories in real-time for multiple robots, taking into account their trajectory tracking dynamics and actuation limits. An event-triggered replanning strategy is proposed to account for disturbances in the system. We adopted the on-demand collision avoidance method presented in previous work to efficiently compute non-colliding trajectories in transition tasks. Preliminary results in simulation show a higher success rate than previous online methods based on Buffered Voronoi Cells (BVC), while maintaining computational tractability for real-time operation.

@MISC{luis-icra19,
author = {Carlos E. Luis and Marijan Vukosavljev and Angela P. Schoellig},
title = {Towards Scalable Online Trajectory Generation for Multi-robot Systems},
year = {2019},
howpublished = {Abstract and Poster, in Proc. of the Resilient Robot Teams Workshop at the IEEE International Conference on Robotics and Automation (ICRA)},
abstract = {We present a distributed model predictive control (DMPC) algorithm to generate trajectories in real-time for multiple robots, taking into account their trajectory tracking dynamics and actuation limits. An event-triggered replanning strategy is proposed to account for disturbances in the system. We adopted the on-demand collision avoidance method presented in previous work to efficiently compute non-colliding trajectories in transition tasks. Preliminary results in simulation show a higher success rate than previous online methods based on Buffered Voronoi Cells (BVC), while maintaining computational tractability for real-time operation.},
}

Knowledge transfer between robots with online learning for enhancing robot performance in impromptu trajectory tracking
S. Zhou, A. Sarabakha, E. Kayacan, M. K. Helwa, and A. P. Schoellig
Abstract and Presentation, in Proc. of the Resilient Robot Teams Workshop at the IEEE International Conference on Robotics and Automation (ICRA), 2019.

As robot dynamics become more complex, learning from data is emerging as an alternative for obtaining accurate dynamic models to assist control system designs or to enhance robot performance. Though being effective, common model learning techniques rely on rich datasets collected from the robots, and the learned experience is often platform-specific. In this work, we propose an online learning approach for transferring deep neural network (DNN) inverse dynamics models across two robots and analyze the role of dynamic similarity in the transfer problem. We demonstrate our proposed knowledge transfer approach with two different quadrotors on impromptu trajectory tracking tasks, in which the quadrotors are required to track arbitrary hand-drawn trajectories accurately from the first attempt. With this work, we illustrate that (i) we can relate the transferability of DNN inverse models to the robot dynamic properties, and (ii) when the transfer is feasible, we can significantly reduce data recollections that would be otherwise costly or risky for robot applications. Given a heterogeneous robot team, we envision having to train only one of the agents to allow the whole team achieving higher performance.

@MISC{zhou-icra19,
author = {Siqi Zhou and Andriy Sarabakha and Erdal Kayacan and Mohamed K. Helwa and Angela P. Schoellig},
title = {Knowledge Transfer Between Robots with Online Learning for Enhancing Robot Performance in Impromptu Trajectory Tracking},
year = {2019},
howpublished = {Abstract and Presentation, in Proc. of the Resilient Robot Teams Workshop at the IEEE International Conference on Robotics and Automation (ICRA)},
abstract = {As robot dynamics become more complex, learning from data is emerging as an alternative for obtaining accurate dynamic models to assist control system designs or to enhance robot performance. Though being effective, common model learning techniques rely on rich datasets collected from the robots, and the learned experience is often platform-specific. In this work, we propose an online learning approach for transferring deep neural network (DNN) inverse dynamics models across two robots and analyze the role of dynamic similarity in the transfer problem. We demonstrate our proposed knowledge transfer approach with two different quadrotors on impromptu trajectory tracking tasks, in which the quadrotors are required to track arbitrary hand-drawn trajectories accurately from the first attempt. With this work, we illustrate that (i) we can relate the transferability of DNN inverse models to the robot dynamic properties, and (ii) when the transfer is feasible, we can significantly reduce data recollections that would be otherwise costly or risky for robot applications. Given a heterogeneous robot team, we envision having to train only one of the agents to allow the whole team achieving higher performance.}
}

Our paper Fast and In Sync: Periodic Swarm Patterns for Quadrotors was nominated for the Best Paper Award on Multi-Robot Systems and the Best Paper Award on Uncrewed Aerial Vehicles, for which Xintong gave a presentation Tuesday, May 21st.

Prof. Schoellig was an organizer of the workshop on Algorithms and Architectures for Learning in-the-Loop Systems in Autonomous Flight, a speaker at the workshop on Resilient Robot Teams: Composing, Acting, and Learning, and a panelist in the Debates on the Future of Robotics Research.

Finally, we also hosted the U of T Robotics booth and the NSERC Canadian Robotics Network booth.

## Schedule

Click the schedule to add items to your calendar. An ics version is also available.