Found 15 results.

a survey of iterative learning control

Author(s): D.A. Bristow, M. Tharayil, A.G. Alleyne
Venue: IEEE Control Systems Magazine (Volume 26, Issue 3)
Year Published: 2006
Keywords: learning from demonstration, survey, nonlinear systems, gaussians
Expert Opinion: The content of the paper provides the reader with a broad perspective of the important ideas, potential, and limitations of iterative learning control - ILC. Besides the design techniques, it discusses problems in stability, performance, learning transient behavior, and robustness.

probabilistic movement primitives

Author(s): Alexandros Paraschos, Christian Daniel, Jan Peters, and Gerhard Neumann
Venue: Neural Information Processing Systems Conference (NeurIPS)
Year Published: 2013
Keywords: manipulation, probabilistic models, gaussians, planning, learning from demonstration
Expert Opinion: This and the following papers using ProMPs, because they provided a very nice formulation for representing probabilistic movement primitives. ProMPs have many advantages and I found them better than classical DMPs in many robotics applications, from gestures to whole-body manipulations.

stanley: the robot that won the darpa grand challenge

Author(s): Sebastian Thrun, Mike Montemerlo, Hendrik Dahlkamp, David Stavens, Andrei Aron, James Diebel, Philip Fong, John Gale, Morgan Halpenny, Gabriel Hoffmann, Kenny Lau, Celia Oakley, Mark Palatucci, Vaughan Pratt, and Pascal Stang
Venue: Journal of Robotic Systems
Year Published: 2006
Keywords: gaussians, state estimation
Expert Opinion: There would not be this much focus on robotics and learning if not for self-driving cars. Self-driving cars would not be a thing without Stanley.

variational inference for policy search in changing situations

Author(s): Gerhard Neumann
Venue: International Conference on Machine Learning
Year Published: 2011
Keywords: reinforcement learning, gaussians
Expert Opinion: Breakthrough work on policy search which preceded much following work on inference-based RL and is totally under appreciated by many authors who have (often knowingly) build upon it. It was kind of direction change in the field but is not given the proper credit.

automatic gait optimization with gaussian process regression

Author(s): Daniel Lizotte, Tao Wang, Michael Bowling, Dale Schuurmans
Venue: International Joint Conference on Artificial Intelligence
Year Published: 2007
Keywords: locomotion, legged robots, gaussians
Expert Opinion: This paper is from the line of papers on Aibo Gate optimization started by Kohl and Stone in 2004. This paper introduced the idea of using Gaussian process regression for learning so as to avoid local optima, make full use of all historical data, and explicitly model noise in gait evaluation. The authors acheived impressive results for optimizing both speed and smoothness with dramatically fewer gait evaluations than prior approaches.

robots that can adapt like animals

Author(s): Antoine Cully, Jeff Clune, Danesh Tarapore, Jean-Baptiste Mouret
Venue: Nature
Year Published: 2015
Keywords: gaussians, probabilistic models, locomotion
Expert Opinion: because it shows how you can leverage models in simulation for learning how to recover from damages, without necessarily re-learning the damaged model. Also, they learn in very few trials on the real robot, which is fundamental when working with real robots and experiments are expensive

on learning, representing and generalizing a task in a humanoid robot

Author(s): Sylvain Calinon, Florent Guenter and Aude Billard
Venue: IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) (Volume 27, Issue 2)
Year Published: 2007
Keywords: humanoid robotics, gaussians
Expert Opinion: First work to explicitly represent motion variance for motion generation.

gaussian processes for data-efficient learning in robotics and control

Author(s): Marc Peter Deisenroth, Dieter Fox, Carl Edward Rasmussen
Venue: IEEE Transactions on Pattern Analysis and Machine Intelligence
Year Published: 2017
Keywords: gaussians, dynamical systems, probabilistic models, reinforcement learning
Expert Opinion: This paper shows the power of model-based reinforcement learning for robot control. It nicely illustrates the power of Gaussian Processes to capture the uncertainty and demonstrates how to leverage it in a highly data-efficient reinforcement learning algorithm. Overall, PILCO (the algorithm described in this paper) might be the most data-efficient algorithm I know. Please note that conference versions of this paper were published at ICML (2011 - PILCO: A model-based and data-efficient approach to policy search) and RSS (2011 - Learning to Control a Low-Cost Manipulator using Data-Efficient Reinforcement Learning).

pattern recognition and machine learning

Author(s): Christopher M. Bishop
Venue: Book
Year Published: 2006
Keywords: probabilistic models, gaussians, unsupervised learning, reinforcement learning
Expert Opinion: A wonderful overview of pattern matching/machine learning minus DNNs.

pilco: a model-based and data-efficient approach to policy search

Author(s): Marc Peter Deisenroth, Carl Edward Rasmussen
Venue: International Conference of Machine Learning
Year Published: 2011
Keywords: state estimation, reinforcement learning, probabilistic models, gaussians, dynamical systems, visual perception, policy gradients
Expert Opinion: The paper by Marc Deisenroth and Carl Rasmussen promotes the use of Gaussian processes (GPs) for model-based reinforcement learning and proposes the PILCO algorithm, one of the most influential algorithms in recent reinforcement learning. GPs are by now heavily used in control and robotics communities. While this paper wasn't the first to use GPs in this context, it's arguably one of the most influential ones. Moreover, this work addresses the problem of data-efficiency in RL, which is of crucial importance for RL in the real world (such as in robotics). The PILCO algorithms has since been used in many different applications and extended in many ways. I consider the PILCO algorithm (or the underlying approach to model-based RL) as one of the state-of-the-art methods in modern RL.

belief space planning assuming maximum likelihood observations

Author(s): Robert Platt Jr., Russ Tedrake, Leslie Kaelbling, Tomas Lozano-Perez
Venue: Robotics: Science and Systems VI
Year Published: 2010
Keywords: manipulation, dynamical systems, planning, gaussians
Expert Opinion: This isn't a learning paper, but a planning paper. Nonetheless, I feel the need to include it, as it strongly influenced my thinking about manipulation learning. It was the first work that I had seen make POMDPs work for real robotics problems. It teaches students that reasoning about uncertainty is important, but that you need to make the right assumptions in order to make it work for any reasonably sized problem. Whereas PILCO aims to reduce model uncertainty, this work assumes a correct model and leverages information-gathering actions to reduce state uncertainty. Thus, these papers are complementary when discussing uncertainty.

motion planning under uncertainty using iterative local optimization in belief space

Author(s): Jur van den Berg, Sachin Patil, Ron Alterovitz
Venue: International Journal of Robotics Research
Year Published: 2012
Keywords: planning, trajectory optimization, dynamical systems, gaussians
Expert Opinion: This paper presents on of the first efficient solutions for continuous PoMDPs. Many follow up papers for solving PoMDPs used similar ideas (i.e., trajectory optimization in belief state). While the application of this algorithm was mainly limited to simulation, solving continuous MDPs is a very important topic for robotics. which I expect to also have much more impact in the future.

generative adversarial nets

Author(s): Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio
Venue: Neural Information Processing Systems Conference (NeurIPS)
Year Published: 2014
Keywords: unsupervised learning, neural networks, gaussians
Expert Opinion: Because it introduces a new way of learning that shows a substantially improved behavior.

model learning for robot control: a survey

Author(s): Duy Nguyen-Tuong, Jan Peters
Venue: Cognitive Science
Year Published: 2011
Keywords: gaussians, survey, dynamical systems, optimal control, unsupervised learning, reinforcement learning
Expert Opinion: The only non-RL paper on my list :). Modelling of robots is part of both very classical control approaches as well as modern learning approaches. There are many excellent papers, I chose this one for providing a wide overview. One of my favourite papers on this topic, by the same authors, included in this survey combines insights from analytic modelling (allowing fast identification of a small set of parameters) with Gaussian process modelling (allowing precise and flexible modelling, but at the cost of requiring more data). I chose this survey instead, as it provides a wider overview and is thus something I would be more likely to suggest to a student or mentee to get a wider overview.

learning to control a low-cost manipulator using data-efficient reinforcement learning

Author(s): Marc Peter Deisenroth, Carl Edward Rasmussen, Dieter Fox
Venue: Robotics: Science and Systems VII
Year Published: 2011
Keywords: manipulation, reinforcement learning, probabilistic models, locomotion, planning, gaussians
Expert Opinion: While this was not the first nor the last publication from Deisenroth and colleagues on PILCO (probabilistic inference for learning control), this is the paper that I remember my colleagues talking about that led me to learn about the approach. This paper was prescient bringing our attention to, and attempting to address, a number of problems in robot learning that still remain important today: data-efficient learning and transfer between related tasks. This paper has a had lasting impact on the field forming the basis of other impressive works in areas of robotics ranging from manipulation to learning underwater swimming gaits.