Author(s): Matthias Hebbel, Ralf Kosse and Walter Nistico
Venue: IEEE-RAS International Conference of Humanoid Robots
Year Published: 2006
Keywords: locomotion, legged robots, genetic algorithms, evolution
Expert Opinion: This paper describes the open loop modelling of a robot gait which mimics the human walking style. The authors develop a parameterized model for the leg and arm motions. They then compare various machine learning methods for finding the best parameters, i.e., the ones that provide the best walk. The paper is very interesting as: - it is one of the pioneer works on robot gait learning - it rises many issues related to practical application of machine learning methods on real hardware - it gives many insights (again, mainly practical) on how to develop a robot learning framework. While the scientific contribution may be limited, the paper has a great importance for its presentation of practical issues. For this reason, I recommend its reading to young students interested in studying this topic for the first time.

Author(s): Craig W. Reynolds
Venue: International Conference on Simulation of Adaptive Behavior
Year Published: 1994
Keywords: genetic algorithms, evolution
Expert Opinion: The work features the automatic synthesis of a symbolic robot controller in a non-deterministic environment via genetic programming. Despite being an early paper on robot learning it features a combination of many aspects that are often not found in modern papers, i.e., (1) learning of explainable, symbolic code, (2) automatic sensor placement, (3) strong non-determinism. Reynolds even goes to great lengths to analyse the code generated by the evolutionary process and identifies a more general framework for how a good solution looks like. Structure and interpretability play an important role in this paper.

Author(s): Karl Sims
Venue: SIGGRAPH
Year Published: 1994
Keywords: dynamical systems, genetic algorithms
Expert Opinion: This paper demonstrated that machine learning and optimization do not have to be restricted to the generation of behavior of a robot. Rather, morphology and shape of an agent can be changed and optimized in an automatic fashion, too. In doing so, the paper created some of the first complex (an extremely impressive and life-like) examples of artificial creatures whose brain and body are fully synthesized. The video accompanying this paper is one of the best research videos out there. The paper has also spawned a number of follow ups, in particular by the group of Hod Lipson at Columbia.

Author(s): L. Yang, C. M. Chew, A. N. Poo, T. Zielinska
Venue: IEEE/RSJ International Conference on Intelligent Robots and Systems
Year Published: 2006
Keywords: locomotion, legged robots, genetic algorithms, planning
Expert Opinion: This paper presents a method for optimally generating stable bipedal walking gaits, based on a Truncated Fourier Series Formulation with coefficients tuned by Genetic Algorithms. It also provides a way to adjust the stride-frequency, step-length or walking pattern in real-time. The proposed approach can be adapted to the robot kinematic structure and to different terrains. As for the my previous suggestion, albeit simple the paper is useful to bridge the gap between robot kinematics (model-based design) and machine learning (model-free design). This is why I recommend it.

Author(s): Richard S. Sutton and Andrew G. Barto
Venue: Book
Year Published: 2018
Keywords: mobile robots, reinforcement learning, unsupervised learning, optimal control, genetic algorithms
Expert Opinion: It presents the definite theoretical basis of reinforcement learning, used widely in robotics.

Author(s): Sonia Chernova, Manuela Veloso
Venue: International Conference on Intelligent Robots and Systems
Year Published: 2004
Keywords: planning, mobile robots, evolution, legged robots, genetic algorithms, locomotion
Expert Opinion: This paper presents a clear and concrete mapping of genetic algorithms to a compelling hardware domain: Sony AIBO walking gait and the RoboCup soccer competition. The AIBO was an example of a platform where parameter tuning by hand is particularly tedious (54 parameters), plus the platform was safe to have "practice‚" on its own overnight (i.e. without human supervision, a rarity for mobile robots)---offering an opportunity for fully autonomous and on-hardware optimization-based learning, where it was feasible for each generation to be evaluated (according to the fitness function) on the actual robot platform without human supervision or intervention. The learned walk that resulted outperformed all hand-tuned and learned walks that participated in (including that which won) the RoboCup 2003 competition. ** This recommendation is getting a bit into the weeds of specific algorithms---not quite sure if the list is planning to go that deep. It's a work I would present in a class, as a great example of a CS algorithm being translated for use on real robot hardware. Again, not quite sure if that sort of categorization fits the bill.