The workshop is dedicated to recent advances in sensorimotor learning for robotics. The development of robots that are able to learn models of themselves and their environments has long been a goal in the robotics, machine learning, and AI communities. However, most current approaches to robot sensing and control are based on strong prior assumptions, which make them brittle to unmodeled dynamics and unexpected changes in the robot body or the environment. Advances in machine learning, including “deep learning”, nonparametric modeling and inference, and reinforcement learning have recently experienced success in deriving models and policies directly from data. For example, in computer vision, deep learning methods, which learn “everything” from data, including low-level features and intermediate representations, have surpassed traditional approaches in accuracy on problems such as object detection and classification. However, incorporating modern machine learning techniques into real-world sensorimotor systems is still challenging. Most real-world sensorimotor control problems are situated in continuous or high-dimensional environments and require real-time interaction, which can be problematic for classical learning techniques. In order to overcome these difficulties, the modeling, learning, and planning components of a fully adaptive decision making system may need significant modifications. This workshop’s goal is to foster discussion on these issues, especially with the participation of the machine learning and computational biology community.
High-level questions to be addressed include, but are not limited to:
- Is it possible to learn the “torque-to-pixels” high-dimensional sensorimotor dynamics of robots or animals directly from the raw data? If not, what prior knowledge is necessary?
- What are the challenges for high-dimensional cross-modal sensorimotor learning in robotics?
- Can cross-model models be learned independently of a task?
- How can we transfer biological insights to robotic systems (and viceversa)?
- Do engineering insights in machine learning and robotics have a biological explanation?
- How can one balance the representation accuracy and the speed of inference? How much data is needed?
- How can online machine learning be used in high-frequency control of real-world systems?
- How can successful supervised or unsupervised learning techniques be used in sensorimotor control problems?
- How can prior knowledge, including expert knowledge, user demonstrations, or distributional assumptions be incorporated into the learning/planning framework?
- How do biological systems deal with modeling, planning, and control under uncertainty?
- What lessons can be learned across disciplines between the control, neuroscience, and reinforcement learning communities, especially in their use of learning models?
|13:30‑14:15||Ben Kuipers (UMich)||"Bootstrap Learning of Real-World Semantics".|
|14:15‑14:35||Fabio Bonsignorio (SSSUP)||An approach to sensory-motor learning based on information driven self-organization and Lie groups|
|14:35‑15:00||Johannes A. Stork (KTH)||"Semantic interpretable PSRs"|
|15:30‑16:00||Sergey Levine (UCB)||Deep Sensorimotor Learning|
|16:00‑16:15||Chelsea Finn (UCB)||“End-to-End Training of Deep Visuomotor Policies”|
|16:15‑16:35||Martin Llofriu (USF)||"Bio-Inspired Multi-Scale Representation for Navigation Learning"|
|16:35‑17:05||Russ Salakhutdinov (U Toronto)|