Image Segmentation Semantic Communication over Internet of Vehicles Qiang Pan Haonan Tong Jie Lv Tao Luo Zhilong Zhang

2025-05-08 0 0 3.57MB 7 页 10玖币

侵权投诉

Image Segmentation Semantic Communication over

Internet of Vehicles

Qiang Pan, Haonan Tong, Jie Lv, Tao Luo, Zhilong Zhang,

Changchuan Yin, and Jianfeng Li

Beijing Laboratory of Advanced Information Network,

Beijing Key Laboratory of Network System Architecture and Convergence,

Beijing University of Posts and Telecommunications, Beijing, China 100876.

Emails: {pq569375378, hntong, lvj, tluo, ccyin, lijf}@bupt.edu.cn, zhilong.zhang@outlook.com

Abstract—In this paper, the problem of semantic-based efﬁcient

image transmission is studied over the Internet of Vehicles (IoV).

In the considered model, a vehicle shares massive amount of

visual data perceived by its visual sensors to assist other vehicles

in making driving decisions. However, it is hard to maintain a

high reliable visual data transmission due to the limited spectrum

resources. To tackle this problem, a semantic communication

approach is introduced to reduce the transmission data amount

while ensuring the semantic-level accuracy. Particularly, an image

segmentation semantic communication (ISSC) system is proposed,

which can extract the semantic features from the perceived images

and transmit the features to the receiving vehicle that reconstructs

the image segmentations. The ISSC system consists of an encoder

and a decoder at the transmitter and the receiver, respectively.

To accurately extract the image semantic features, the ISSC

system encoder employs a Swin Transformer based multi-scale

semantic feature extractor. Then, to resist the wireless noise and

reconstruct the image segmentation, a semantic feature decoder

and a reconstructor are designed at the receiver. Simulation results

show that the proposed ISSC system can reconstruct the image

segmentation accurately with a high compression ratio, and can

achieve robust transmission performance against channel noise,

especially at the low signal-to-noise ratio (SNR). In terms of mean

Intersection over Union (mIoU), the ISSC system can achieve

an increase by 75%, compared to the baselines using traditional

coding methods.

Index Terms—Image segmentation, semantic communication,

Swin Transformer.

I. INTRODUCTION

In the emerging Internet of Vehicles (IoV), there is a massive

amount of visual information required to be transmitted among

the vehicles to assist driving decisions, which consumes a

certain amount of bandwidth resources [1], [2]. However,

due to the restricted spectrum resources and the complicated

communication conditions in the trafﬁc environment, it is

difﬁcult to maintain a reliable connection to transmit large

amounts of visual data. To this end, semantic communication

[3], [4], with transmitting the data at a semantic level, rather

than the symbol level, is becoming a viable solution to solve

the above challenges. In current IoV system, vehicles mainly

use visual data for locating and obstacle avoidance, it is

available for the receiving vehicle to only reconstruct the image

segmentation for making driving decisions. According to our

research, although semantic communication can reduce the data

transmission amount thus reducing the occupied spectrum, the

image segmentation semantic communication (ISSC) system

has not been well designed.

Existing works [5]–[9] have systematically studied semantic

communication. In [5], the authors introduced a preliminary

theory of semantic information based on logical probabilistic

ranking. The work in [6] presented a model theory based tech-

nique for semantic data compression and trustworthy seman-

tic communication with quantitative measurements. Besides,

the authors in [7] modeled the communication problem as a

Bayesian game to minimize the end-to-end average semantic

metric in a dynamic communication scenario. Furthermore,

the work in [8] developed a deep learning based semantic

communication system for text transmission, which recovered

the meaning of sentences, rather than the transmitted bits

or symbols. Recently, a semantic communication system for

transmitting audio in the Internet of Things (IoT) is proposed,

and federated learning is employed to improve the precision of

semantic data extraction [9]. However, these aforementioned

works did not explore image semantic communication, and

image semantic communication is challenging due to the com-

plexity of image structure.

Along with the development of deep learning, the prior arts

have investigated image semantic communication systems [10]–

[13]. In [10], the authors proposed an end-to-end joint source-

channel coding (JSCC) image system on the structure of au-

toencoder, which provided a smooth performance degradation

with the decrease of signal-to-noise ratio (SNR). Based on

JSCC, the work in [11] exploited the channel feedback for

image transmission, and provided considerable improvements

in terms of the end-to-end image reconstruction quality. Be-

sides, the work in [12] developed a framework for semantic

communication with artiﬁcial intelligence tasks and built a

system for inspecting surface defects of workpieces, which

realized a visual semantic communication. Moreover, The au-

thors in [13] extended the semantic communication for image

classiﬁcation tasks to UAV aerial photography and achieved the

tradeoff between transmission delay and classiﬁcation accuracy.

However, the works in [10]–[13] were mainly concerned with

how to reconstruct or classify the images accurately but did not

propose a system to reconstruct the image segmentation from

semantic features at the receiver directly.

In this paper, a novel ISSC system is proposed for the IoV.

arXiv:2210.05321v1 [cs.NI] 11 Oct 2022

Fig. 1. Single-link vehicle-to-vehicle scenario with an ISSC system over the

IoV.

To our best knowledge, this is the ﬁrst semantic communication

system to reconstruct image segmentation. The main contribu-

tions are as follows:

•We propose an ISSC system that enables the sharing of

visual data over the IoV. The ISSC system consists of an

encoder that extracts the image semantic features at the

transmitter and a decoder which reconstructs the image

segmentation at the receiver.

•We propose a cascade structure to extract multi-scale se-

mantic features of image based on Swin Transformer at the

ISSC encoder. The multi-scale semantic features are ﬁrst

aggregated and then sent to the receiver through a wireless

channel. At the ISSC decoder, the received semantic

features are decoded to reconstruct image segmentation,

which reduces the transmission data and achieves reliable

transmission.

•We evaluate the ISSC system performance through sim-

ulation experiments. Simulation results demonstrate that

the proposed ISSC system can provide robust transmis-

sion performance against channel variation, particularly

at low SNR, and can reconstruct the image segmentation

accurately with a high compression ratio. Compared to

the traditional methods, the proposed ISSC system can

improve mean Intersection over Union (mIoU) by 75%.

The rest of this paper is structured as follows. In Section II,

the system model and problem formulation are presented. We

provide a thorough explanation of the proposed ISSC encoder

and decoder in Section III. In Section IV, the simulation results

are shown and discussed, and Section V concludes the paper.

II. SYSTEM MODEL

We consider a single-link vehicle-to-vehicle image commu-

nication scenario where a front vehicle needs to transmit visual

data to a rear vehicle that deploy an ISSC system over the IoV,

as shown in Fig. 1. Due to the blocking of the front vehicle,

the rear vehicle can not see the obstacle. In this scenario, the

front vehicle extracts and aggregates the semantic features of

the image taken by the camera and sent to the rear vehicle.

Fig. 2. The framework of the proposed ISSC system.

Image segmentation has the category and location information

of objects, so that the rear vehicle can reconstruct the image

segmentation to make driving decisions.

The above ISSC system can be simpliﬁed as shown in

Fig. 2. In the system, the transmitter (front vehicle) sends

image semantic features to the receiver (rear vehicle) through

a wireless channel. The system consists of two components:

(i) the ISSC encoder at the transmitter side, which extracts

semantic features from the input image for transmission; (ii) the

ISSC decoder at the receiver side, which decodes the received

semantic features and utilize them to reconstruct the image

segmentation.

A. ISSC Encoder

The ISSC encoder extracts and aggregates the semantic fea-

tures from the input image S∈RH×W×3through a multi-scale

semantic feature extractor and a semantic feature aggregator,

where H,W, and 3 are the image width, the image height, and

the number of channels. The input image is RGB format and

every value range is in [0,255] which needs to be normalized

to [0,1] by the normalization layer. To extract the semantic

features from the shallow to the deep, the processed image is

put into to a multi-scale semantic feature extractor made up

of neural networks. Then the semantic feature aggregator fuses

the multi-scale semantic features and send them.

We simplify the ISSC encoder parameter as α, thus, the

relationship between the transmitted semantic features xand

the input image Scan be given by

x=Tα(S),(1)

where Tα(·)indicates the function of the ISSC encoder.

When being transmitted over a wireless channel, the encoded

semantic features will suffer channel fading and noise. We

consider the condition of a single communication link, where

the received semantic features yat the ISSC decoder can be

characterized as

y=h·x+ρ,(2)

where his the channel covariance coefﬁcient, and ρ∼

N0, σ2Iis the Gaussian channel noise with variance σ2and

Iis the identity matrix.

B. ISSC Decoder

The ISSC decoder is used to decode the received semantic

features yand reconstruct the image segmentation b

S, consists

of a semantic feature decoder and a reconstructor. First, the

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

ImageSegmentationSemanticCommunicationoverInternetofVehiclesQiangPan,HaonanTong,JieLv,TaoLuo,ZhilongZhang,ChangchuanYin,andJianfengLiBeijingLaboratoryofAdvancedInformationNetwork,BeijingKeyLaboratoryofNetworkSystemArchitectureandConvergence,BeijingUniversityofPostsandTelecommunications,Beijing,China...

展开>> 收起<<

Image Segmentation Semantic Communication over Internet of Vehicles Qiang Pan Haonan Tong Jie Lv Tao Luo Zhilong Zhang.pdf

共7页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Image Segmentation Semantic Communication over Internet of Vehicles Qiang Pan Haonan Tong Jie Lv Tao Luo Zhilong Zhang

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: