Author(s): Takayuki Osa, Joni Pajarinen, Gerhard Neumann, J. Andrew Bagnell, Pieter Abbeel, Jan Peters
Venue: Foundations and Trends in Robotics
Year Published: 2018
Keywords: survey, learning from demonstration, reinforcement learning, planning
Expert Opinion: A focused overview of imitation learning and perspectives from some of the leaders in the field. Not a complete review but an excellent highlighting of important contributions in the field and perspective on future challenges.

Author(s): D.A. Bristow, M. Tharayil, A.G. Alleyne
Venue: IEEE Control Systems Magazine (Volume 26, Issue 3)
Year Published: 2006
Keywords: learning from demonstration, survey, nonlinear systems, gaussians
Expert Opinion: The content of the paper provides the reader with a broad perspective of the important ideas, potential, and limitations of iterative learning control - ILC. Besides the design techniques, it discusses problems in stability, performance, learning transient behavior, and robustness.

Author(s): Stefan Schaal
Venue: Trends in Cognitive Sciences
Year Published: 1999
Keywords: neural networks, humanoid robotics, survey, learning from demonstration, planning
Expert Opinion: I think this work is a seminal work on imitation learning, which is a fundamental framework in the field of robot learning.

and

Author(s): Atkeson, C. G., Moore, A. W., and Schaal, S.
Venue: Artificial Intelligence Review
Year Published: 1997
Keywords: survey
Expert Opinion: This paper is an overview of (rather) early results in model learning for control. Beyond the particular results presented in the paper, which are already impressive and could perhaps be used to calibrate claims in current research in robot learning, the paper provides a thoughtful discussion and important insights on issues related to model learning for controlling mechanical systems from data generation issues to inverse/forward models, representations, etc. Certainly a must read for a compelling vision on robot learning.

Author(s): Duy Nguyen-Tuong, Jan Peters
Venue: Cognitive Science
Year Published: 2011
Keywords: gaussians, survey, dynamical systems, optimal control, unsupervised learning, reinforcement learning
Expert Opinion: The only non-RL paper on my list :). Modelling of robots is part of both very classical control approaches as well as modern learning approaches. There are many excellent papers, I chose this one for providing a wide overview. One of my favourite papers on this topic, by the same authors, included in this survey combines insights from analytic modelling (allowing fast identification of a small set of parameters) with Gaussian process modelling (allowing precise and flexible modelling, but at the cost of requiring more data). I chose this survey instead, as it provides a wider overview and is thus something I would be more likely to suggest to a student or mentee to get a wider overview.

Author(s): Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage, Anil Anthony Bharath
Venue: IEEE Signal Processing Magazine
Year Published: 2017
Keywords: survey, policy gradients, neural networks, reinforcement learning
Expert Opinion: This comprehensive survey greatly covers the field of deep reinforcement learning approaches and algorithms. It is written to be accessible to the wide audience and generally easy to understand.

Author(s): Leslie Pack Kaelbling, Michael L. Littman, Andrew W. Moore
Venue: Journal of Artificial Intelligence Research
Year Published: 1996
Keywords: neural networks, survey, reinforcement learning, probabilistic models
Expert Opinion: This work provides a relatively short and easy to understand introduction to Reinforcement Learning. Although rather old and therefore does not cover the new approaches to reinforcement learning, it covers the problem of RL very well. I usually ask beginning students interested in reinforcement learning to read this paper together with the more recent "Reinforcement Learning in Robotics: A Survey" by Jens Kober, Andrew Bagnell, and Jan Peters, and deep learning approaches to reinforcement learning.

Author(s): Lorenzo Jamone, Emre Ugur, Angelo Cangelosi, Luciano Fadiga, Alexandre Bernardino, Justus Piater and Jose Santos-Victor
Venue: IEEE Transactions on Cognitive and Developmental Systems
Year Published: 2018
Keywords: survey, visual perception, mobile robots, reinforcement learning
Expert Opinion: Affordances is an important term for robot learning, but also one that tends to be overloaded and can lead to confusion. If an object allows an agent to perform an action, then the object is said to afford the action to that agent. Affordances can generally be learned autonomously and are thus a fundamental aspect of self-supervised learning for autonomous robots. The nuances of the term however are still widely discussed in robotics and other fields. As a result, one should be aware of the ambiguity and different perspectives regarding the term when talking about affordances. This survey paper discusses some of the nuanced interpretations of the term affordances.

Author(s): Marc Peter Deisenroth, Gerhard Neumann, Jan Peters
Venue: Book
Year Published: 2013
Keywords: survey, reinforcement learning
Expert Opinion: A great unifying view on policy search

Author(s): Jeannette Bohg, Antonio Morales, Tamim Asfour, Member, Danica Kragic
Venue: IEEE Transactions on Robotics (Volume 30, Issue 2)
Year Published: 2016
Keywords: survey, visual perception, manipulation
Expert Opinion: A must read for anyone interested in grasping. Great survey on data-driving methods for grasping of, both known and unknown, objects; plus a review and connections to 'traditional' analytical methods!

Author(s): Jens Kober, J. Andrew Bagnell, Jan Peters
Venue: International Journal of Robotics Research
Year Published: 2014
Keywords: survey, reinforcement learning, learning from demonstration, optimal control, mobile robots
Expert Opinion: This survey was published at a time when there was still a significant gap between reinforcement learning and its practical employment on real robot hardware. For the majority of real world domains, rollouts are impractical to perform on actual hardware---because, for example, the state/action spaces are continuous, exploration can be dangerous, and rollouts take much longer when physically executed---plus often simulators are too dissimilar to the real world and hardware for what is learned to transfer well. To get reinforcement learning to be effective on a real hardware system, therefore, the devil is in the details, and this article addresses just that. Today the gap is narrowing, in part because of advances in computation, but also because of implementation "tricks‚" becoming codified. This article is a bit of a one-stop-shop for pulling together a lot of these tricks, and putting some theoretical rigor and thought behind why and when they work.

Author(s): Minoru Asada, Koh Hosoda, Yasuo Kuniyoshi, Hiroshi Ishiguro, Toshio Inui, Yuichiro Yoshikawa, Masaki Ogino, and Chisato Yoshida
Venue: IEEE Transactions on Autonomous Mental Development
Year Published: 2009
Keywords: survey, humanoid robotics, cognitive sciences, visual perception
Expert Opinion: I really like the overview of cognitive robotics, where learning can be applied and what the required parts are essential to learning and cognitive (artificial) systems.

Author(s): Stefan Schaal, Christopher G. Atkeson
Venue: IEEE Robotics & Automation Magazine
Year Published: 2010
Keywords: survey, reinforcement learning, policy gradients, optimal control, trajectory optimization
Expert Opinion: This review from Schaal and Atkeson does an excellent job of concisely covering the many approaches to learning control in robotics. It is useful not only as an overview of this subtype of robot learning, but also as a jumping off point for further research, as the works cited are extensive. This paper is also of note because it considers the problem of robot learning from a control perspective, rather than the more common computer science or statistical perspectives. The authors also discuss the practical aspects of learning control, such as the robustness of learned control policies to unexpected perturbation.

Author(s): Oliver Kroemer, Scott Niekum, George Konidaris
Venue: arXiv
Year Published: 2019
Keywords: survey, probabilistic models, manipulation, reinforcement learning
Expert Opinion: This paper present an incredibly extensive recent survey on learning in robot manipulation (440 citations!!). Surveys are always use especially for new grad students. This one presents a single framework to formalise the robot manipulation problem.