Chapter 1: Developmental robotics

The study of the mechanisms, architectures, and constraints that enable embodied machines to learn new skills and knowledge throughout their lifetimes is the goal of developmental robotics (DevRob), also known as epigenetic robotics. Self-exploration of the world and social interaction are hypothesized to yield learning that is cumulative and progressively more complex, just as it is in human children. Beginning with theories of human and animal development elaborated in fields like developmental psychology, neuroscience, developmental and evolutionary biology, and linguistics, researchers typically formalize and implement these theories in robots, sometimes exploring extensions or variants of these theories. Developmental robotics not only provides feedback and novel hypotheses on theories of human and animal development, but also allows researchers to confront those models in a realistic setting through experimentation in robots.

While similar, developmental robotics stands apart from its evolutionary counterpart (ER). While ER makes use of populations of robots that evolve over time, DevRob is more concerned with how the structure of a single robot's control system changes as a result of experience.

The fields of robotics and artificial life are also connected to the DevRob framework.

Is it possible for a robot to pick up new skills as easily as a kid? Can it acquire new abilities and knowledge in a dynamic setting that was not fully specified during development? How can it learn about itself and how it fits into its natural and social surroundings? Once it is "out of the factory," how can its cognitive abilities be allowed to grow without the help of an engineer? What can it pick up from observing people in their natural social environments? Developmental robotics seeks to answer these fundamental questions. Although Alan Turing and other cybernetics pioneers posed these questions and outlined a general approach as early as 1950, it wasn't until the latter part of the 20th century that they were actually studied systematically.

Developmental robotics is related to areas like AI and ML as well as cognitive robotics and computational neuroscience because of its emphasis on adaptive intelligent machines. While it may make use of some of the methods developed in these areas, it is distinct from them in a number of important ways. Embodied and situated sensorimotor and social skills are prioritized over abstract symbolic problems, setting it apart from classical AI in that it does not assume the capability of advanced symbolic reasoning. In contrast to cognitive robotics, it is concerned with the processes rather than the final products of cognitive development. Functional modeling of integrated architectures of development and learning is its main focus, setting it apart from computational neuroscience. In a broader sense, the following three characteristics distinguish developmental robotics from other related fields::

It aims for architectures and learning mechanisms that are task-agnostic; that is, the machine or robot should be able to pick up tasks for which the engineer has no blueprints; Open-ended growth and continuous education are emphasized, i.e.

the ability of a living organism to continually learn new things.

This should not be understood as a capacity for learning "anything" or even "everything", it's just that there are infinite ways to build upon the foundation of skills one acquires; Acquired expertise should progressively increase in complexity (while remaining manageable).

Embodied AI, enactive and dynamical systems cognitive science, and connectionism all played a role in the emergence of developmental robotics. The field of developmental robotics strongly interacts with others, including developmental psychology, developmental and cognitive neuroscience, drobotics, and others, because it is based on the fundamental idea that learning and development occur as the self-organized result of the dynamical interactions among brains, bodies, and their physical and social environment. Because many of the theories in these fields are verbal and/or descriptive, developmental robotics requires significant work in formalization and computational modeling. These computational models are then used to do a number of things, including evaluate their coherence and possibly explore alternative explanations for understanding biological development, all with the goal of creating more versatile and adaptive machines.

Using the same general approach and methodology as human infants, developmental robotics projects aim to have robots learn the same skills. The development of sensorimotor abilities is one of the first areas of study. Learning to use a tool requires an understanding of its affordances as well as an understanding of one's own body's structure and dynamics, such as hand-eye coordination, locomotion, and interaction with objects. The second set of abilities that developmental robots aim to instill are social and linguistic ones, such as turn-taking, coordinated interaction, lexicons, syntax, and grammar, and their foundation in sensorimotor abilities (sometimes referred as symbol grounding). Concurrently, the development of self-awareness and other-awareness, the maturation of attentional capacities, the maturation of categorization systems and higher-level representations of affordances or social constructs, and the maturation of values, empathy, and theories of mind are all being studied.

Due to the vastness and complexity of the sensorimotor and social spaces in which humans and robots operate, only a fraction of the possible skills can be explored and learned in a single lifetime. Therefore, developmental organisms require mechanisms and constraints to direct their growth in complexity. Developmental robotics studies several significant families of guiding mechanisms and constraints that are modeled after human development:

There are two primary types of motivational systems, each of which generates internal reward signals that drive exploration and learning:

Robots and living organisms are pushed by extrinsic motivations to keep certain essential internal properties stable, such as their supply of food and water, their health, and their exposure to light (e.g. in phototropic systems); Curiosity-driven learning and exploration, also known as active learning and exploration, is the result of a robot's intrinsic motivations to seek out novelty, challenge, compression, or learning progress in and of itself; Developmental robotics explores mechanisms that can allow robots to participate in social interaction in ways that are analogous to those through which humans learn a great deal. This could pave the way for robots to learn from humans (via methods as varied as imitation, emulation, stimulus enhancement, demonstration, etc.) and for robots to trigger natural human pedagogy through their understanding of social cues. As a result, the topic of people's opinions on developmental robots is also studied; Confidence intervals Learning efficiency can often be greatly improved by studying and removing the biases that characterize representations/encodings and inference mechanisms. Another crucial area of study is the neural mechanisms underlying the inference and acquisition of new knowledge and skills through the reuse of previously learned structures; Learning new sensorimotor or social skills can be greatly facilitated by the properties of embodiment, such as geometry, materials, or innate motor primitives/synergies often encoded as dynamical systems. An important direction of inquiry concerns how these constraints interact with one another; Infants have developmental limitations because their bodies and nervous systems must mature separately rather than coming together at birth. This suggests, for instance, that as learning and development progress, new degrees of freedom may appear, along with increases in the volume and resolution of available sensorimotor signals. One of the most pressing issues in developmental robotics is how to implement these mechanisms in robots so as to facilitate or impede the learning of new complex skills.

While the majority of developmental robotics projects involve extensive interactions with theories of animal and human development, the levels of similarity and inspiration between identified biological mechanisms and their counterpart in robots, as well as the abstraction levels of modeling, may vary greatly. Unlike Neurorobotics, which seeks to model both the function and the biological implementation (neural or morphological models), other projects may only focus on functional modeling of the mechanisms and constraints described above. These projects may, for instance, reuse techniques from applied mathematics or engineering fields within their architectures.

Due to the fact that developmental robotics is both a new and ambitious area of study, there are still many important questions that need to be answered.

To begin, current methods are far from enabling high-dimensional robots in the real world to acquire a limitless repertoire of ever-more-complex skills over the course of their lifetimes. An important problem to be addressed is the high dimensionality of continuous sensorimotor spaces. Acquiring new knowledge and skills is a lifelong pursuit. Despite having brains and morphologies which are immensely more powerful than existing computational mechanisms, no experiments lasting more than a few days have been set up so far. This starkly contrasts with the time required for human infants to learn basic sensorimotor skills.

The interaction between the mechanisms and constraints described in the preceding section shall be investigated more systematically as one of the strategies to...