Online Feedback Equilibrium Seeking
Giuseppe Belgioioso*, Dominic Liao-McPherson*, Mathias Hudoba de Badyn,
Saverio Bolognani, Roy S. Smith, John Lygeros, and Florian D¨
orfler
Abstract—This paper proposes a unifying design framework
for dynamic feedback controllers that track solution trajectories
of time-varying generalized equations, such as local minimizers
of nonlinear programs or competitive equilibria (e.g., Nash) of
non-cooperative games. Inspired by the feedback optimization
paradigm, the core idea of the proposed approach is to re-purpose
classic iterative algorithms for solving generalized equations
(e.g., Josephy–Newton, forward-backward splitting) as dynamic
feedback controllers by integrating online measurements of
the continuous-time nonlinear plant. Sufficient conditions for
closed-loop stability and robustness of the algorithm-plant cyber-
physical interconnection are derived in a sampled-data setting
by combining and tailoring results from (monotone) operator,
fixed-point, and nonlinear systems theory. Numerical simulations
on smart building automation and competitive supply-chain
management are presented to support the theoretical findings.
I. INTRODUCTION
Online feedback optimization (FO) [1] is an emerging con-
trol paradigm for optimal steady-state operation of complex
systems based on their direct closed-loop interconnection with
optimization algorithms. FO controllers can handle control
objectives beyond set-point regulation, typically tracking (a-
priori unknown) solution trajectories of time-varying con-
strained optimization problems. In recent years, FO controllers
have been proposed for a wide variety of problem settings [1]–
[7]. These can be categorized by the type of control objective
(e.g., convex or nonconvex) and constraints (e.g., hard or
soft), the dynamics of the plant (e.g., nonlinear, linear, or
algebraic), the type of algorithm (discrete or continuous-time),
and the stability analysis (e.g., continuous-time, discrete-time,
or hybrid), see [1] for a comprehensive list. FO has found
widespread application in various domains, including power
systems (e.g., for optimal power reserve dispatch [2], [3], or
frequency regulation in AC grids [4], [5]), communication
networks (e.g., for network congestion control [6]), and trans-
portation systems (e.g., for ramp metering control [7]).
These large-scale engineering infrastructures comprise mul-
tiple subsystems with local decision authority and preferences,
commonly known as agents. These agents are typically self-
interested, hence, a more general notion of “efficiency” is
needed to model desirable (i.e., safe, locally optimal, and
strategically stable) operating points for such systems, motivat-
ing the use of game-theoretic equilibria (e.g., Nash, Wardrop)
[8]. A timely example are modern supply-chain systems,
*These authors contributed equally to this work. G. Belgioioso, S. Bolog-
nani, R. Smith, J. Lygeros, and F. D¨
orfler are with the ETH Z¨
urich
Automatic Control Laboratory, {gbelgioioso, bsaverio, rsmith,
jlygeros, dorfler}@ethz.ch. D. Liao-McPherson is with the Uni-
versity of British Columbia, dliaomcp@mech.ubc.ca. M. Hudoba de
Badyn is with the University of Oslo, mathias.hudoba@its.uio.no.
This work is supported by the SNSF via NCCR Automation (Grant 180545).
whereas suppliers, manufacturers, and retailers compete over
the available resources (e.g., raw materials, market demand)
to maximize their local profits [9], [10].
There are several classes of control approaches for driving
non-cooperative multi-agent systems to steady-state configura-
tions given by game-theoretic equilibria. One class uses first-
order algorithms and builds on operator-theoretic and passivity
arguments [11]–[13]. Passivity is leveraged in [11] to design
a distributed feedback control law to drive agents with single-
integrator dynamics to a steady-state operating point given by
a Nash equilibrium (NE). This approach is extended to multi-
integrator agents affected by partially-known disturbances in
[12]. The case of games with coupling constraints and agents
with mixed-order integrator dynamics is considered for the first
time in [7]. All the these works consider agents coupled via
their objective functions (and constraints) but with decoupled
dynamics and use continuous-time flows.
A second class of methods are sampled-data approaches
based on the model-free extremum seeking (ES) framework,
for finding optima [14], game-theoretic equilibria [15], [16],
and solutions to dynamic inclusions (which subsume the two
previous cases) [17], [18]. In [15], an ES controller is designed
for NE seeking in games in which the agents have nonlinear
decoupled dynamics. The extension to games with coupling
constraints is presented in [16]. In both cases, practical stabil-
ity is proven in the disturbance-free case. In [17], a general
framework is developed for the design and analysis of a class
ES controllers applicable to optimization as well as non-
cooperative games. Practical stability is established by relying
on the mathematical framework of hybrid dynamic inclusions.
To accelerate convergence, [18] extends the previous frame-
work from periodic to event-triggered sampled-data control.
These methods require nominal stability of the plant and,
typically, time-scale separation assumptions.
A third class of methods are extensions of economic model
predictive control to non-cooperative systems, including multi-
objective MPC [19] and various flavours of game-theoretic
MPC [20], [21]. Unlike the first-order or ES methods, these
consider transient operation and can handle unstable systems.
In exchange, they require dynamic models of the plant and
are typically computationally heavier due to the need to find
accurate equilibria of trajectory games at each sampling time.
In this paper, we propose feedback equilibrium seeking
(FES), an extension of FO that seeks to drive pre-stabilized
dynamical systems to “efficient” operating points encoded by
time-varying generalized equations (GEs). GEs contain con-
strained optimization as a special case and can model a broad
range of equilibrium problems (e.g., Nash, Wardrop). FES con-
trollers are most similar to the first class of control approaches
discussed above [11]–[13]. They are typically based on first-
arXiv:2210.12088v3 [math.OC] 14 Feb 2024