Music in Motion: Dancing Quadrocopters | Dynamic Systems Lab

Music in Motion: Dancing Quadrocopters

Home » Research » Aerial, Applications, Control, Robotics » Music in Motion: Dancing Quadrocopters

← Previous Next →

This project features agile, multi-vehicle flight performances that are designed and executed to music. We develop motion planning, control and learning algorithms that result in collision-free, perfectly timed flight motions. Enjoy!

The original project page at ETH Zurich with more details is found here.

Related Publications

Summary and Overview Papers

So you think you can dance? Rhythmic flight performances with quadrocopters
A. P. Schoellig, H. Siegel, F. Augugliaro, and R. D’Andrea
in Controls and Art, A. LaViers and M. Egerstedt, Eds., Springer international publishing, 2014, pp. 73-105.

This chapter presents a set of algorithms that enable quadrotor vehicles to “fly with the music”; that is, to perform rhythmic motions that are aligned with the beat of a given music piece.

@INCOLLECTION{schoellig-springer14,
author = {Angela P. Schoellig and Hallie Siegel and Federico Augugliaro and Raffaello D'Andrea},
title = {So you think you can dance? {Rhythmic} flight performances with quadrocopters},
booktitle = {{Controls and Art}},
editor = {Amy LaViers and Magnus Egerstedt},
publisher = {Springer International Publishing},
pages = {73-105},
year = {2014},
doi = {10.1007/978-3-319-03904-6_4},
urldata={../../wp-content/papercite-data/data/schoellig-springer14-files.zip},
urlslides={../../wp-content/papercite-data/slides/schoellig-springer14-slides.pdf},
urllink = {http://www.tiny.cc/MusicInMotionSite},
urlvideo={https://www.youtube.com/playlist?list=PLD6AAACCBFFE64AC5},
abstract = {This chapter presents a set of algorithms that enable quadrotor vehicles to "fly with the music"; that is, to perform rhythmic motions that are aligned with the beat of a given music piece.}
}

Dance of the flying machines: methods for designing and executing an aerial dance choreography
F. Augugliaro, A. P. Schoellig, and R. D’Andrea
IEEE Robotics Automation Magazine, vol. 20, iss. 4, pp. 96-104, 2013.

Imagine a troupe of dancers flying together across a big open stage, their movement choreographed to the rhythm of the music. Their performance is both coordinated and skilled; the dancers are well rehearsed, and the choreography well suited to their abilities. They are no ordinary dancers, however, and this is not an ordinary stage. The performers are quadrocopters, and the stage is the ETH Zurich Flying Machine Arena, a state-of-the-art mobile testbed for aerial motion control research.

@ARTICLE{augugliaro-ram13,
author = {Federico Augugliaro and Angela P. Schoellig and Raffaello D'Andrea},
title = {Dance of the Flying Machines: Methods for Designing and Executing an Aerial Dance Choreography},
journal = {{IEEE Robotics Automation Magazine}},
volume = {20},
number = {4},
pages = {96-104},
year = {2013},
doi = {10.1109/MRA.2013.2275693},
urlvideo={http://youtu.be/NRL_1ozDQCA?t=21s},
urlslides={../../wp-content/papercite-data/slides/augugliaro-ram13-slides.pdf},
abstract = {Imagine a troupe of dancers flying together across a big open stage, their movement choreographed to the rhythm of the music. Their performance is both coordinated and skilled; the dancers are well rehearsed, and the choreography well suited to their abilities. They are no ordinary dancers, however, and this is not an ordinary stage. The performers are quadrocopters, and the stage is the ETH Zurich Flying Machine Arena, a state-of-the-art mobile testbed for aerial motion control research.}
}

Periodic Motion Planning, Control and Learning

Generation of collision-free trajectories for a quadrocopter fleet: a sequential convex programming approach
F. Augugliaro, A. P. Schoellig, and R. D’Andrea
in Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2012, pp. 1917-1922.

This paper presents an algorithm that generates collision-free trajectories in three dimensions for multiple vehicles within seconds. The problem is cast as a non-convex optimization problem, which is iteratively solved using sequential convex programming that approximates non-convex constraints by using convex ones. The method generates trajectories that account for simple dynamics constraints and is thus independent of the vehicle’s type. An extensive a posteriori vehicle-specific feasibility check is included in the algorithm. The algorithm is applied to a quadrocopter fleet. Experimental results are shown.

@INPROCEEDINGS{augugliaro-iros12,
author = {Federico Augugliaro and Angela P. Schoellig and Raffaello D'Andrea},
title = {Generation of collision-free trajectories for a quadrocopter fleet: A sequential convex programming approach},
booktitle = {{Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}},
pages = {1917-1922},
year = {2012},
doi = {10.1109/IROS.2012.6385823},
urlvideo = {https://youtu.be/wwK7WvvUvlI?list=PLD6AAACCBFFE64AC5},
abstract = {This paper presents an algorithm that generates collision-free trajectories in three dimensions for multiple vehicles within seconds. The problem is cast as a non-convex optimization problem, which is iteratively solved using sequential convex programming that approximates non-convex constraints by using convex ones. The method generates trajectories that account for simple dynamics constraints and is thus independent of the vehicle's type. An extensive a posteriori vehicle-specific feasibility check is included in the algorithm. The algorithm is applied to a quadrocopter fleet. Experimental results are shown.}
}

Feed-forward parameter identification for precise periodic quadrocopter motions
A. P. Schoellig, C. Wiltsche, and R. D’Andrea
in Proc. of the American Control Conference (ACC), 2012, pp. 4313-4318.

This paper presents an approach for precisely tracking periodic trajectories with a quadrocopter. In order to improve temporal and spatial tracking performance, we propose a feed-forward strategy that adapts the motion parameters sent to the vehicle controller. The motion parameters are either adjusted on the fly or, in order to avoid initial transients, identified prior to the flight performance. We outline an identification scheme that tunes parameters for a large class of periodic motions, and requires only a small number of identification experiments prior to flight. This reduced identification is based on analysis and experiments showing that the quadrocopter’s closed-loop dynamics can be approximated by three directionally decoupled linear systems. We show the effectiveness of this approach by performing a sequence of periodic motions on real quadrocopters using the tuned parameters obtained by the reduced identification.

@INPROCEEDINGS{schoellig-acc12,
author = {Angela P. Schoellig and Clemens Wiltsche and Raffaello D'Andrea},
title = {Feed-forward parameter identification for precise periodic quadrocopter motions},
booktitle = {{Proc. of the American Control Conference (ACC)}},
pages = {4313-4318},
year = {2012},
doi = {10.1109/ACC.2012.6315248},
urlvideo = {http://tiny.cc/MusicInMotion},
urlslides = {../../wp-content/papercite-data/slides/schoellig-acc12-slides.pdf},
abstract = {This paper presents an approach for precisely tracking periodic trajectories with a quadrocopter. In order to improve temporal and spatial tracking performance, we propose a feed-forward strategy that adapts the motion parameters sent to the vehicle controller. The motion parameters are either adjusted on the fly or, in order to avoid initial transients, identified prior to the flight performance. We outline an identification scheme that tunes parameters for a large class of periodic motions, and requires only a small number of identification experiments prior to flight. This reduced identification is based on analysis and experiments showing that the quadrocopter's closed-loop dynamics can be approximated by three directionally decoupled linear systems. We show the effectiveness of this approach by performing a sequence of periodic motions on real quadrocopters using the tuned parameters obtained by the reduced identification.}
}

Feasibility of motion primitives for choreographed quadrocopter flight
A. P. Schoellig, M. Hehn, S. Lupashin, and R. D’Andrea
in Proc. of the American Control Conference (ACC), 2011, pp. 3843-3849.

This paper describes a method for checking the feasibility of quadrocopter motions. The approach, meant as a validation tool for preprogrammed quadrocopter performances, is based on first principles models and ensures that a desired trajectory respects both vehicle dynamics and motor thrust limits. We apply this method towards the eventual goal of using parameterized motion primitives for expressive quadrocopter choreographies. First, we show how a large class of motion primitives can be formulated as truncated Fourier series. We then show how the feasibility check can be applied to such motions by deriving explicit parameter constraints for two particular parameterized primitives. The predicted feasibility constraints are compared against experimental results from quadrocopters in the ETH Flying Machine Arena.

@INPROCEEDINGS{schoellig-acc11,
author = {Angela P. Schoellig and Markus Hehn and Sergei Lupashin and Raffaello D'Andrea},
title = {Feasibility of motion primitives for choreographed quadrocopter flight},
booktitle = {{Proc. of the American Control Conference (ACC)}},
pages = {3843-3849},
year = {2011},
doi = {10.1109/ACC.2011.5991482},
urlvideo = {https://www.youtube.com/playlist?list=PLD6AAACCBFFE64AC5},
urlslides = {../../wp-content/papercite-data/slides/schoellig-acc11-slides.pdf},
urldata = {../../wp-content/papercite-data/data/schoellig-acc11-files.zip},
abstract = {This paper describes a method for checking the feasibility of quadrocopter motions. The approach, meant as a validation tool for preprogrammed quadrocopter performances, is based on first principles models and ensures that a desired trajectory respects both vehicle dynamics and motor thrust limits. We apply this method towards the eventual goal of using parameterized motion primitives for expressive quadrocopter choreographies. First, we show how a large class of motion primitives can be formulated as truncated Fourier series. We then show how the feasibility check can be applied to such motions by deriving explicit parameter constraints for two particular parameterized primitives. The predicted feasibility constraints are compared against experimental results from quadrocopters in the ETH Flying Machine Arena.}
}

A platform for dance performances with multiple quadrocopters
A. P. Schoellig, F. Augugliaro, and R. D’Andrea
in Proc. of the Workshop on Robots and Musical Expressions at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2010, pp. 1-8.

This paper presents a platform for rhythmic flight with multiple quadrocopters. We envision an expressive multimedia dance performance that is automatically composed and controlled, given a random piece of music. Results in this paper prove the feasibility of audio-motion synchronization when precisely timing the side-to-side motion of a quadrocopter to the beat of the music. An illustration of the indoor flight space and the vehicles shows the characteristics and capabilities of the experimental setup. Prospective features of the platform are outlined and key challenges are emphasized. The paper concludes with a proof-of-concept demonstration showing three vehicles synchronizing their side-to-side motion to the music beat. Moreover, a dance performance to a remix of the sound track ‘Pirates of the Caribbean’ gives a first impression of the novel musical experience. Future steps include an appropriate multiscale music analysis and the development of algorithms for the automated generation of choreography based on a database of motion primitives.

@INPROCEEDINGS{schoellig-iros10,
author = {Angela P. Schoellig and Federico Augugliaro and Raffaello D'Andrea},
title = {A platform for dance performances with multiple quadrocopters},
booktitle = {{Proc. of the Workshop on Robots and Musical Expressions at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}},
pages = {1-8},
year = {2010},
urlvideo = {https://youtu.be/aaaGJKnJdrg?list=PLD6AAACCBFFE64AC5},
urlvideo2 = {https://www.youtube.com/playlist?list=PLD6AAACCBFFE64AC5},
urlslides = {../../wp-content/papercite-data/slides/schoellig-iros10-slides.pdf},
abstract = {This paper presents a platform for rhythmic flight with multiple quadrocopters. We envision an expressive multimedia dance performance that is automatically composed and controlled, given a random piece of music. Results in this paper prove the feasibility of audio-motion synchronization when precisely timing the side-to-side motion of a quadrocopter to the beat of the music. An illustration of the indoor flight space and the vehicles shows the characteristics and capabilities of the experimental setup. Prospective features of the platform are outlined and key challenges are emphasized. The paper concludes with a proof-of-concept demonstration showing three vehicles synchronizing their side-to-side motion to the music beat. Moreover, a dance performance to a remix of the sound track 'Pirates of the Caribbean' gives a first impression of the novel musical experience. Future steps include an appropriate multiscale music analysis and the development of algorithms for the automated generation of choreography based on a database of motion primitives.}
}

Synchronizing the motion of a quadrocopter to music
A. P. Schoellig, F. Augugliaro, and R. D’Andrea
in Proc. of the IEEE International Conference on Robotics and Automation (ICRA), 2010, pp. 3355-3360.

This paper presents a quadrocopter flying in rhythm to music. The quadrocopter performs a periodic side-to-side motion in time to a musical beat. Underlying controllers are designed that stabilize the vehicle and produce a swinging motion. Synchronization is then achieved by using concepts from phase-locked loops. A phase comparator combined with a correction algorithm eliminate the phase error between the music reference and the actual quadrocopter motion. Experimental results show fast and effective synchronization that is robust to sudden changes in the reference amplitude and frequency. Changes in frequency and amplitude are tracked precisely when adding an additional feedforward component, based on an experimentally determined look-up table.

@INPROCEEDINGS{schoellig-icra10,
author = {Angela P. Schoellig and Federico Augugliaro and Raffaello D'Andrea},
title = {Synchronizing the motion of a quadrocopter to music},
booktitle = {{Proc. of the IEEE International Conference on Robotics and Automation (ICRA)}},
pages = {3355-3360},
year = {2010},
doi = {10.1109/ROBOT.2010.5509755},
urlslides = {../../wp-content/papercite-data/slides/schoellig-icra10-slides.pdf},
urlvideo = {https://youtu.be/Kx4DtXv_bPo?list=PLD6AAACCBFFE64AC5},
abstract = {This paper presents a quadrocopter flying in rhythm to music. The quadrocopter performs a periodic side-to-side motion in time to a musical beat. Underlying controllers are designed that stabilize the vehicle and produce a swinging motion. Synchronization is then achieved by using concepts from phase-locked loops. A phase comparator combined with a correction algorithm eliminate the phase error between the music reference and the actual quadrocopter motion. Experimental results show fast and effective synchronization that is robust to sudden changes in the reference amplitude and frequency. Changes in frequency and amplitude are tracked precisely when adding an additional feedforward component, based on an experimentally determined look-up table.}
}

Experimental Platform

A platform for aerial robotics research and demonstration: The Flying Machine Arena
S. Lupashin, M. Hehn, M. W. Mueller, A. P. Schoellig, and R. D’Andrea
Mechatronics, vol. 24, iss. 1, pp. 41-54, 2014.

The Flying Machine Arena is a platform for experiments and demonstrations with fleets of small flying vehicles. It utilizes a distributed, modular architecture linked by robust communication layers. An estimation and control framework along with built-in system protection components enable prototyping of new control systems concepts and implementation of novel demonstrations. More recently, a mobile version has been featured at several eminent public events. We describe the architecture of the Arena from the viewpoint of system robustness and its capability as a dual-purpose research and demonstration platform.

@ARTICLE{lupashin-mech14,
author = {Sergei Lupashin and Markus Hehn and Mark W. Mueller and Angela P. Schoellig and Raffaello D'Andrea},
title = {A platform for aerial robotics research and demonstration: {The Flying Machine Arena}},
journal = {{Mechatronics}},
volume = {24},
number = {1},
pages = {41-54},
year = {2014},
doi = {10.1016/j.mechatronics.2013.11.006},
urllink = {http://flyingmachinearena.org/},
urlvideo={https://youtu.be/pcgvWhu8Arc?list=PLuLKX4lDsLIaVjdGsZxNBKLcogBnVVFQr},
abstract = {The Flying Machine Arena is a platform for experiments and demonstrations with fleets of small flying vehicles. It utilizes a distributed, modular architecture linked by robust communication layers. An estimation and control framework along with built-in system protection components enable prototyping of new control systems concepts and implementation of novel demonstrations. More recently, a mobile version has been featured at several eminent public events. We describe the architecture of the Arena from the viewpoint of system robustness and its capability as a dual-purpose research and demonstration platform.}
}