1 INTERACTIVE INFERENCE A MUL TI -AGENT MODEL OF COOPERATIVE JOINT ACTIONS

2025-04-30 0 0 1.57MB 32 页 10玖币

侵权投诉

INTERACTIVE INFERENCE: A MULTI-AGENT MODEL OF

COOPERATIVE JOINT ACTIONS

Domenico Maisto, Francesco Donnarumma, and Giovanni Pezzulo

Institute for Cognitive Sciences and Technologies, National Research Council, Rome, Italy

Contact: Giovanni Pezzulo (giovanni.pezzulo@istc.cnr.it)

Abstract—We advance a novel computational model of multi-agent, cooperative joint

actions that is grounded in the cognitive framework of active inference. The model assumes

that to solve a joint task, such as pressing together a red or blue button, two (or more) agents

engage in a process of interactive inference. Each agent maintains probabilistic beliefs about

the joint goal (e.g., should we press the red or blue button?) and updates them by observing

the other agent’s movements, while in turn selecting movements that make his own

intentions legible and easy to infer by the other agent (i.e., sensorimotor communication).

Over time, the interactive inference aligns both the beliefs and the behavioral strategies of the

agents, hence ensuring the success of the joint action. We exemplify the functioning of the

model in two simulations. The first simulation illustrates a “leaderless” joint action. It shows

that when two agents lack a strong preference about their joint task goal, they jointly infer it

by observing each other’s movements. In turn, this helps the interactive alignment of their

beliefs and behavioral strategies. The second simulation illustrates a "leader-follower" joint

action. It shows that when one agent (“leader”) knows the true joint goal, it uses sensorimotor

communication to help the other agent (“follower”) infer it, even if doing this requires

selecting a more costly individual plan. These simulations illustrate that interactive inference

supports successful multi-agent joint actions and reproduces key cognitive and behavioral

dynamics of “leaderless” and "leader-follower" joint actions observed in human-human

experiments. In sum, interactive inference provides a cognitively inspired, formal framework

to realize cooperative joint actions and consensus in multi-agent systems.

Keywords: active inference, consensus, joint action, multi-agent systems, sensorimotor

communication, shared knowledge, social interaction.

I. INTRODUCTION

A central challenge of multi-agent systems (MAS) is coordinating the actions of multiple

autonomous agents in time and space, to accomplish cooperative tasks and achieve joint goals

[1], [2]. Developing successful multi-agent systems requires addressing controllability

challenges [3], [4] and dealing with synchronization control [5], formation control [6], task

allocation [7] and consensus formation [8]–[10].

Research in cognitive science may provide guiding principles to address the above

challenges, by identifying the cognitive strategies that groups of individuals use to

successfully interact with each other and to make collective decisions [11]–[14]. An extensive

body of research studied how two or more people coordinate their actions in time and space

during cooperative (human-human) joint actions, such as when performing team sports,

dancing or lifting something together [15], [16]. These studies have shown that successful

joint actions engage various cognitive mechanisms, whose level of sophistication plausibly

depends on task complexity. The simplest forms of coordination and imitation in pairs or

groups of individuals, such as the joint execution of rhythmic patterns, might not require

sophisticated cognitive processing, but could use simple mechanisms of behavioral

synchronization – perhaps based on coupled dynamical systems, analogous to the

synchronization of coupled pendulums [17]. However, more sophisticated types of joint

actions go beyond the mere alignment of behavior. For example, some joint actions require

making decisions together, e.g., the decision about where to place a table that we are lifting

together. These sophisticated forms of joint actions and joint decisions might benefit from

cognitive mechanisms for mutual prediction, mental state inference, sensorimotor

communication and shared task representations [16], [18]. The cognitive mechanisms

supporting joint action have been probed by numerous experiments [19]–[29], sometimes

with the aid of conceptual [30], computational [31]–[39], and robotic [40]–[43] models.

However, there is still a paucity of models that implement advanced cognitive abilities, such

as the inference of others' plans and the alignment of task knowledge across group members,

which have been identified in empirical studies of joint action. Furthermore, it is unclear

whether and how it is possible to develop joint action models from first principles; for

example, from the perspective of a generic inference or optimization scheme that unifies

multiple cognitive mechanisms required for joint action.

We advance an innovative framework for cooperative joint action and consensus in multi-

agent systems, inspired by the cognitive framework of active inference. Active inference is a

normative theory that describes the brain as a prediction machine, which learns an internal

(generative) model of the statistical regularities of the environment – including the statistics

of social interactions – and uses it to generate predictions that guide perceptual processing

and action planning [44]. Here, we use the predictive and inferential mechanisms of active

inference to implement sophisticated forms of joint action in dyads of interacting agents. The

model presented here formalizes joint action as a process of interactive inference based on

shared task knowledge between the agents [2], [45]. We exemplify the functioning of the

model in a “joint maze” task. In the task, two agents have to navigate in a maze, to reach and

press together either a red or a blue button. Each agent has probabilistic beliefs about the

joint task that the dyad is performing, which covers his own and the other agent's

contributions (e.g., should we both press a red or a blue button?). Each agent continuously

infers what the joint task is, based on his (stronger or weaker) prior belief and the

observation of the other agent's movements towards one of the two buttons. Then, he selects

an action (red or blue button press), in a way that simultaneously fulfills two objectives. The

former, pragmatic objective consists in achieving the joint task efficiently (e.g., by following

the shortest route to reach the to-be-pressed button). The latter, epistemic objective consists

in shaping one's movements to help the other agent inferring what the joint goal is (e.g., by

selecting a longer route easily associated to the goal of pressing the red button).

The next sections are organized as follows. First, we introduce the consensus problem

(called a “joint maze”) we will use throughout the paper to explain and validate our approach.

Next, we illustrate the main tenets of the interactive inference model of joint action. Then, we

present two simulations that illustrate the functioning of the interactive inference model. The

first simulation shows that over time, the interactive inference aligns the joint task

representations of the two agents and their behavior, as observed empirically in several joint

action studies [18], [23], [46]–[49]. In turn, this form of “interactive alignment” (or

“generalized synchrony”) optimizes the performance of the dyad. The second simulation

shows that when agents have asymmetric information about the joint task, the more

knowledgeable agent (or "leader") systematically modifies his behavior, to reduce the

uncertainty of the less knowledgeable agent (or "follower"), as observed empirically in

studies of sensorimotor communication [16], [18]. This form of “social epistemic action”

ensures the success of joint actions despite incomplete information. Finally, we discuss how

our model of interactive inference could help better understand various facets of ("leaderless"

and "leader-follower") human joint actions, by providing a coherent formal explanation of

their dynamics at both brain and behavioral levels.

II. PROBLEM FORMULATION AND SCENARIO

To illustrate the mechanisms of the interactive inference model, we focus on the consensus

problem called “joint maze” task, which closely mimics the setting used in a previous human

joint action study [50], see Fig. 1. In this task, two agents (represented as a grey hand and a

white hand) have to “navigate” in a grid-like maze, reach the location in which the red or blue

button is located, and then press it together. The task is completed successfully when both

agents “press” the same button, whatever its color (unless stated otherwise).

Fig. 1. Schematic illustration of the “joint maze” task. The two (grey and white) agents are

represented as two hands. Their initial positions are L3 (grey) and L19 (white). Their possible

goal locations are in blue (L12) and red (L10). The agents can navigate in the maze, by

following the open paths, but cannot go through walls).

At the beginning of each simulation, each agent is equipped with some prior knowledge (or

preference) about the goal of the task. This prior knowledge is represented as a probabilistic

belief, i.e., a probability distribution over four possible task states; these are “both agents will

press red”, “both agents will press blue”, “the white agent will press red and the grey agent

will press blue” and “the white agent will press blue and the grey agent will press red”.

Importantly, in different simulations, the prior knowledge of the two agents can be congruent

(if both assign the highest probability to the same state) or incongruent (if they assign the

highest probability to different states); certain (if the probability mass is peaked in one state)

or uncertain (if the probability mass is spread across all the states). This creates a variety of

coordination problems, which span from easy (e.g., if the beliefs of the two agents are

congruent and certain) to difficult problems (e.g., if the beliefs are incongruent or uncertain).

Each simulation includes several trials, during which each agent follows a perception-action

cycle. First, the agent receives an observation about his own position and the position of the

other agent. Then, he updates his knowledge about the goal of the task (i.e., joint task

inference) and forms a plan about how to reach it (i.e., joint plan inference). Finally, he makes

one movement in the maze (by sampling it probabilistically from the plan that he formed).

Then, a new perception-action cycle starts.

III. METHODS

Here, we provide a brief introduction to the active inference framework for single agents (see

[44] for details) and then we illustrate the novel, interactive inference model developed here

to address multi-agent, cooperative joint actions.

A. Active Inference

Active Inference agents implement perception and action planning through the

minimization of variational free energy [44]. To minimize free energy, the agents use a

generative model of the causes of their perceptions, which encodes the joint probability of the

stochastic variables illustrated in Fig. 2, using the formalism of probabilistic graphical models

[51].

Fig. 2. Generative model of an active inference agent, unrolled in time. The circles denote

stochastic variables; filled circles denote observed variables, whereas unfilled circles denote

variables that are not observed and need to be inferred. The squares indicate the categorical

probability distributions that parameterize the generative model. Please see the main text for

a detailed explanation of the variables.

The agent's generative model is defined as follows:







(1)

where , , ,

, .

The set  parameterizes the generative model. The (likelihood) matrix A

encodes the relations between the observations  and the hidden causes of observations

. The (transition) matrix B defines how hidden states evolve over time , as a function of a

control state (action) ; note that a sequence of control states  defines a policy

 (see below for a definition). The matrix  is an a-priori probability distribution over

observations and encodes the agent's preferences or goals. The matrix  is the prior belief

about the initial hidden state, before the agent receives any observation. Finally,  is a

precision that regulates action selection and is sampled from a  distribution with parameters

 and .

An active inference agent implements the perception-action loop by applying the above

matrices to hidden states and observations. In this perspective, perception corresponds to

estimating hidden states on the basis of observations and of previous hidden states. At the

beginning of the simulation, the agent has access through D to an initial state estimate  and

receives an observation  that permits refining the estimate by using the likelihood matrix A.

Then, for , the agent infers its current hidden state  based on the

observations previously collected and by considering the transitions determined by the

control state , as specified in B. Specifically, active inference uses an approximate posterior

over (past, present and future) hidden states and parameters . Assuming a mean

field approximation, it can be described as:





 (2)

where the sufficient statistics are encoded by the expectations 

, with 







. Following a variational approach, the distribution in Eq. (2) best approximates the

posterior when its sufficient statistics  minimise the free energy of the generative model, see

[44]. This condition holds when the sufficient statistics are:





 (3.a)

 (3.b)

 

 (3.c)

where the symbol “” denotes the inner product, defined as , with the two

arbitrary matrices  and .

Action selection is operated by selecting the policy (i.e., sequence of control states

) that is expected to minimize free energy the most in the future. The policy

distribution  is expressed in (3.b); the term  is a softmax function,  encodes a prior over

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

1INTERACTIVEINFERENCE:AMULTI-AGENTMODELOFCOOPERATIVEJOINTACTIONSDomenicoMaisto,FrancescoDonnarumma,andGiovanniPezzuloInstituteforCognitiveSciencesandTechnologies,NationalResearchCouncil,Rome,ItalyContact:GiovanniPezzulo(giovanni.pezzulo@istc.cnr.it)Abstract—Weadvanceanovelcomputationalmodelofmulti-a...

展开>> 收起<<

1 INTERACTIVE INFERENCE A MUL TI -AGENT MODEL OF COOPERATIVE JOINT ACTIONS.pdf

共32页,预览5页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

1 INTERACTIVE INFERENCE A MUL TI -AGENT MODEL OF COOPERATIVE JOINT ACTIONS

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: