Networked Signal and Information Processing Stefan Vlaski Soummya Kar Ali H. Sayed and Jos e M. F. Moura Abstract

2025-05-02 0 0 2.67MB 32 页 10玖币

侵权投诉

Networked Signal and Information Processing

Stefan Vlaski, Soummya Kar, Ali H. Sayed, and Jos´e M. F. Moura

Abstract

Abstract. The article reviews signiﬁcant advances in networked signal and information process-

ing, which have enabled in the last 25 years extending decision making and inference, optimiza-

tion, control, and learning to the increasingly ubiquitous environments of distributed agents. As

these interacting agents cooperate, new collective behaviors emerge from local decisions and ac-

tions. Moreover, and signiﬁcantly, theory and applications show that networked agents, through

cooperation and sharing, are able to match the performance of cloud or federated solutions, while

oﬀering the potential for improved privacy, increasing resilience, and saving resources.

1 Introduction

Since its beginnings, throughout the past century and still dominant at the turn of the 21st

century, the signal and information processing (SIP) prevailing paradigm has been to process

signals and information by stand-alone systems or central computing units, with no cooperation

or interaction among them, see left of Fig. 1. This focus has led to tremendous progress in a wide

range of problems in speech and image processing, control and guidance, estimation and ﬁltering

theories, communications theory, and array processing, with enormous impact in telecommu-

nication and wireless, audio, medical imaging, multimedia, radar, and other application areas.

In the nearly 25 years since the turn of the century, each of these areas has progressed rapidly,

in large part due to increases in computational resources along with the availability of data,

giving rise to a variety of advanced data-driven processing tools. At the end of the century,

we also witnessed signiﬁcant technological progress from massive layouts of ﬁber at the back-

bone, to successes in high speed wireless and wiﬁ deployments, to chip advances combining in a

single miniature inexpensive platform functionalities like sensing, storage, communication, and

computing, and to breakthroughs in network protocols and software. This progress has led for

example to the launching of hundreds, soon thousands and millions, of inexpensive sensing de-

vices (we will call here agents) that sense, compute, communicate and are networked, ushering a

paradigm shift in SIP. Initially, the agents observed data independently of one another and sim-

ply forwarded their raw data to the cloud, with no local processing in a centralized architecture,

see Fig. 1. Parallel architectures soon emerged where agents started processing their local data,

transferring only their (local) inferences to a fusion center. The fusion center aggregated the

arXiv:2210.13767v2 [eess.SP] 18 Apr 2023

Non-cooperative Centralized or parallel Federated Networked or decentralized

Figure 1: Taxonomy of networked multi-agent systems.

locally processed data and orchestrated the computations that occurred in parallel at the indi-

vidual agents. While traditionally computation and communication occurred in a synchronous

fashion, synchrony requirements were relaxed, like with federated learning architectures, third

from left in Fig. 1. But as a result of scenarios with abundant data available at dispersed net-

worked locations, such as sensor networks that monitor large geographical regions, or robotic

swarms that collaborate over a common task, or social networks of many interacting agents,

a new critical trend started to materialize. This led to decentralization and democratization

of technology and, towards the middle and end of the ﬁrst decade of this century, signal and

information processing moved deﬁnitely from parallel, federated, or edge architectures,1to a

distributed,decentralized, or networked paradigm. The agents sense and process their own data

and then cooperate with other agents. They transmit to and exchange information with agents

in their vicinity. It marked the appearance of networked elementary processing units, with each

unit collecting and processing data and sharing their information with immediate neighbors.

Individual agents are now capable of local inference decisions and limited actions. The coupling

among the agents gives rise to powerful network structures that open up vast opportunities for

the solution of more complex problems by tapping into the power of the group. Examples of

such networked systems are plentiful, including instrumented critical infrastructures like water,

gas, ﬁnancial networks, smart grids, as well as networked mobile devices, swarms of drones, au-

tonomous vehicles, or populations of individuals. The interconnectedness of the agents within

the network allows for their cooperation to rise from local to global coherent decision and ac-

tion. To study, understand, and steer the occurrence of these global behaviors adds new layers

of complexity. More advanced analytical tools became necessary to combine local processing

with cooperation among the agents. This ushered the design of new processing algorithms,

new methods to derive performance guarantees and assess their quality, to examine the eﬀect

of agents coupling on network stability, to endow agents with adaptation and learning abilities

1We interpret an edge architecture as a layered or hierarchical federated architecture.

and with the capacity to handle privacy, and to enable such networks to contribute to multiple

tasks at once. Distributed,decentralized, or networked architectures achieve aggregation and

coordination through device-to-device or peer-to-peer interactions. Computation is no longer

at the cloud or like in federated or edge computing at a fusion center, but fully distributed at

the device level. Synchrony requirements need not be assumed. Networked architectures may

be viewed as a generalization of centralized and federated conﬁgurations, allowing us to recover

federated algorithms from distributed or decentralized ones by employing a star-topology.

Networked distributed processing architectures are more robust—if an edge or an agent

fails, the entire network can continue to process data and deliver inference decisions. There is

no need for costly communications with the cloud or a remote edge server. Furthermore, while

the exchange of processed iterates might still leak some private information, recent works have

demonstrated that networked architectures can be designed to oﬀer improved privacy due to

their decentralized nature [1–4] Even more importantly, distributed networked architectures can

be shown to match the performance of centralized solutions.

This tutorial article surveys the recent history of networked signal and information process-

ing including consensus and diﬀusion strategies for regression problems [5–9] developed in the

2000s, detection and parameter estimation over networks [10–14] and their performance guar-

antees [12–16], distributed optimization [17–30], learning, and adaptation [29–31]. It provides

a comprehensive coverage of topics and references. We will bridge the gap by unifying under a

common umbrella more recent applications to distributed machine learning including multitask

learning [32] and nonconvex optimization [33,34], design variants under diﬀerent operating sce-

narios such as asynchronous algorithms [35] and connections to alternative architectures such

as federated learning [36]

2 Historical remarks

There has been extensive work on distributed techniques for information and signal processing

over networks. Many optimal inference problems adopt a quadratic optimization cost whose

solution, under linear models and Gaussian noise, is a linear statistic of the data. With peer-to-

peer communication among sensors, the question becomes how to compute the global average

of the local statistics only through cooperation among the agents. Departing from centralized

architectures, the solution built on the consensus strategy for distributed averaging, with no need

for a fusion center to collect the dispersed data for processing. Consensus was initially proposed

by DeGroot [37] to enable a group of agents to reach agreement by pooling their information

and to converge [37, 38] to an average estimate solely by interaction among neighbors. Many

subsequent works were devoted to characterizing consensus’ convergence behavior, the role of the

graph topology, random selection of neighbors, and several other aspects. Some sample works

include [9, 39–42], while useful overviews appear in [11, 31] with many additional references.

Several works in the literature proposed extensions of the original consensus construction in

order to more generally minimize aggregate cost functions, such as mean-square-error costs, or

to solve distributed estimation problems of the least-squares or Kalman ﬁltering type. These

extensions involve constructions with gradient-descent type updates. Among these works we

may mention [40, 43–46]. While an early version of the consensus gradient-based algorithm for

distributed estimation and optimization already appears in [40], convergence results were limited

to the asymptotic regime and there was no understanding of the performance of the algorithm, its

actual convergence rate, and the inﬂuence of data, topology, quantization, noise, and asynchrony

on behavior. These considerations are of great signiﬁcance when designing practical, data-driven

systems and they attracted the attention of the signal processing community after the turn of

the century. Moreover, some of the earlier investigations on consensus implementations involved

separate time scales (fast communication and consensus iterations among agents, slow data

collection), which can be a challenge for streaming or online data.

Online consensus implementations where data is collected at every consensus step, appeared

in the works by [9, 12, 17, 47, 48] and others. Using decaying step-sizes, these works established

the ability of the algorithms to converge. In particular, the work [12] introduced the so-called

consensus+innovations variant, which responds to streaming data and established several per-

formance measures in terms of convergence rate, and the eﬀect of topology, quantization, and

noisy conditions and other factors—see also [48]. In parallel with these developments, online

distributed algorithms of the diﬀusion type were introduced by [7, 49–51] to enable continuous

adaptation and learning by networked agents under constant step-size conditions. The diﬀusion

strategies modiﬁed the consensus update in order to incorporate symmetry, which was shown

to enlarge the stability range of the network and to enhance its performance, even under decay-

ing step-sizes—see [52] and the overviews [29, 31]. The diﬀusion structure was used in several

subsequent works for distributed optimization such as [20, 53–55] and other works.

In all these and related works on online distributed inference, the goal is for every agent in

the network to converge to an estimate of the unknown by relying exclusively on local interac-

tions with its neighbors. Important questions that arise in this context include: 1) convergence:

do the distributed inference algorithms converge and if so in what sense; 2) agreement: do the

agents reach a consensus on their inferences; 3) distributed versus centralized: how good is the

distributed inference solution at each agent when compared with the centralized inference ob-

tained by a fusion center, in other words are the distributed inference sequences consistent, and

asymptotically unbiased, eﬃcient, and normal; and 4) rate of convergence: what is the rate at

which the distributed inference at each agent converges. These questions require very diﬀerent

approaches than the methods used in the “consensus or averaging only” solution from earlier

works. Solutions that emerged of the consensus and diﬀusion type combine at each iteration

1) an aggregation step that fuses the current inference statistic at each agent with the states of

their neighbors, with 2) a local update driven by the new observation at the agent. This generic

framework, of which there are several variations, is very diﬀerent from the standard consensus

where in each time step only local averaging of the neighbors’ states occurs, and no observations

are processed, and from other distributed inference algorithms with multiple time-scales, where

between-measurement updates involve a large number of consensus steps (theoretically, an inﬁ-

nite number of steps). The classes of successful distributed inference algorithms that emerged

add only to the identiﬁability condition of the centralized model that the network be connected

on average. The results for these algorithms are also shown to hold under broad conditions like

agents’ communication channel intermittent failures, asynchronous and random communication

protocols, and quantized communication (limited bandwidth), making their application realistic

when 1) a large number of agents are involved (bound to fail at random times), 2) packet losses

in wireless digital communications cause links to fail intermittently, 3) agents communicate

asynchronously and 4) the agents may be resource constrained and have a limited bit budget

for communication. Furthermore, these distributed inference algorithms make no distributional

assumptions on the agents and link failures that can be spatially correlated. Readers may refer

to the overviews [11,29, 31] and the many references therein.

There have of course been many other useful contributions to the theory and practice of net-

worked information and signal processing, with many other variations and extensions. However,

space limitations prevent us from elaborating further. Readers can refer to the overviews [11,31].

Among such extensions, we may mention extensive works on distributed Kalman ﬁltering

by [43, 56–59] and others. Other parts of this manuscript refer to other classes of distributed

algorithms, such as constructions of the primal and primal-dual type, and the corresponding

references. The presentation actually presents a uniﬁed view of a large family of distributed

implementations, including consensus and diﬀusion, for online inference by networked agents.

Notation. All vectors are column vectors. We employ bold font to emphasize random

variables, and regular font for their realizations as well as deterministic quantities. Upper case

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

NetworkedSignalandInformationProcessingStefanVlaski,SoummyaKar,AliH.Sayed,andJoseM.F.MouraAbstractAbstract.Thearticlereviewssignicantadvancesinnetworkedsignalandinformationprocess-ing,whichhaveenabledinthelast25yearsextendingdecisionmakingandinference,optimiza-tion,control,andlearningtotheincreasi...

展开>> 收起<<

Networked Signal and Information Processing Stefan Vlaski Soummya Kar Ali H. Sayed and Jos e M. F. Moura Abstract.pdf

共32页,预览5页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Networked Signal and Information Processing Stefan Vlaski Soummya Kar Ali H. Sayed and Jos e M. F. Moura Abstract

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: