The last years have witnessed the soaring of Machine Learning (ML), which has provided disruptive performance gains in several fields. Apart from undeniable advances in methodology, these gains are often attributed to massive amounts of training data and computing power, which led to breakthroughs in speech recognition, computer vision and natural language processing. In this project, we propose to extend these advances to sequential decision making of multiple agents for planning and control. We particularly target learning realistic behavior with multiple horizons, requiring long-term planning at the same time as short-term fine-grained control. Obtaining gains through learning from massive amounts of data is not as easy as in supervised learning, as learning to control requires to learn from interactions instead of learning from static data collections.

Control Theory (CT) is the second key methodology of this project, together with ML. The two fields, while being distinct, have a long history of interactions between them and as both fields mature, their overlap is more and more evident. CT aims to provide differential model-based approaches to solve stabilization and estimation problems. These model-driven approaches are powerful because they are based on a thorough understanding of the system and can leverage established physical relationships. However, nonlinear models usually need to be simplified, they have difficulty accounting for noisy data and non modeled uncertainties. At some level, they are limited by the complexity (strong nonlinearities, big dimension) of the real systems. For linear differential equations, the theory is both complete and constructive, and we can guarantee that the design will provide the required tools as well as the limits of the models and actuators allow it.

Machine Learning, on the other hand, aims at learning complex models from (often large amounts of) data and can provide data-driven models for a wide range of tasks. Markov Decision Processes (MDP) and Reinforcement Learning (RL) have traditionally provided a mathematically founded framework for control applications, where agents are required to learn policies from past interactions with an environment. In recent years, this methodology has been combined with deep neural networks, which play the role of high-capacity function approximators, and model the discrete or continuous policy function or a function of the cumulated reward of the agent, or both.

This dichotomy between design and learning is typical of AI on its overlap with other domains and is widely discussed in, e.g., computer vision (handcrafted vs. learned feature representations) language (linguistics vs. learning), statistical predictions (handcrafted causal models vs. data driven predictors) etc. While in many applications learning has become the prevailing methodology, process control is still a field where control engineering cannot be replaced for many low level control problems, mainly due to (i) lack of stability of learned controllers, and (ii) computational complexity.

DeLiCio proposes fundamental research on the crossroads of ML/IA and CT with planned algo- rithmic contributions on the integration of models, prior knowledge and learning in control and the perception action cycle: data-driven learning / identification of physical models for control; stability and robustness priors for reinforcement learning; stable decentralized (multi-agent) control using ML and CT. The planned methodological advances of this project will be evaluated on a challenging appli- cation requiring planning as well as fine-grained control, namely the synchronization of a UAV swarm through learning. The objective is to learn strategies which allow a swarm to solve a high-level goal (navigation, visual search) while at the same time maintaining a formation.

Organization

DeLiCio will last 4 years. It is structured into 4 interacting work-packages, plus a project management package WP0. WP1 and WP2 address single agent problems and propose different ways of integrating ML and CT. Building on their results, WP3 addresses multi-agent behavior and WP4 provides a challenging application of the scientific advances to the control of UAV fleets.

Selected Results

Hybrid system identification with deep learning + control theory (CDC 2021; HAL+under review)

Observer design and deep learning (CDC 2021)

Estimating UAV generated turbulences from onboard vision (arxiv, under review)

Machine learning of complex physical phenomena (ICLR 2022)

Hybrid control (CDC 2020)

Impact

Scientific impact

The scientific results obtained by project DeLiCio are of fundamental nature and concern a large number of potential problems in machine learning, in process control and in robotics. We predict a large impact of the addressed contributions (hybrid control, statistical model reduction, stability and robustness priors etc.) on RL agents, making them (i) more sample efficient and thus to be more widely applicable, and (ii) less computational complex and thus easier to implement in embedded systems.

Impact on society

Although our contributions will be evaluated on the control of UAV fleets, extensions may include industrial and service robotics, ADAS systems, distributed control of multi-agent systems such as fleet of autonomous vehicles/cars (eg. speed consensus of autonomous cars on motorways), communication reductions in networked control systems, and even AI for games, household devices and smart devices. The results of project DeLiCio could bring society a big step further towards smart ambient intelligence.

Open access and open data

We strongly believe that science advances best and quickest through open protocols. Reference code implementations will be made available with each publication. When applicable, we will publish learned models online, including trained parameters of deep networks.