1 Self-supervised Learning for Clustering of Wireless Spectrum Activity

2025-04-28 0 0 3.89MB 15 页 10玖币

侵权投诉

Self-supervised Learning for Clustering of Wireless

Spectrum Activity

Ljupcho Milosheski∗†, Gregor Cerar∗, Blaˇ

z Bertalaniˇ

c∗, Carolina Fortuna∗, Mihael Mohorˇ

ciˇ

c∗†

∗Department of Communication Technologies, Jozef Stefan Institute, Jamova 39, 1000, Ljubljana, Slovenia

†Jozef Stefan International Postgraduate School, Jamova 39, 1000, Ljubljana, Slovenia

Email: {ljupcho.milosheski, miha.mohorcic, carolina.fortuna}@ijs.si

Abstract—In recent years, much work has been done on

processing of wireless spectrum data involving machine learn-

ing techniques in domain-related problems for cognitive radio

networks, such as anomaly detection, modulation classiﬁcation,

technology classiﬁcation and device ﬁngerprinting. Most of the

solutions are based on labeled data, created in a controlled

manner and processed with supervised learning approaches.

However, spectrum data measured in real-world environment

is highly nondeterministic, making its labeling a laborious and

expensive process, requiring domain expertise, thus being one

of the main drawbacks of using supervised learning approaches

in this domain. In this paper, we investigate the utilization of

self-supervised learning (SSL) for exploring spectrum activities

in a real-world unlabeled data. In particular, we assess the

performance of SSL models, based on the reference DeepCluster

architecture. We carefully consider the current state-of-the-art

feature extractors, taking into account the performance and

complexity trade-offs. Our ﬁndings demonstrate that SSL models

achieve superior performance regarding the feature quality and

clustering performance compared to baseline feature learning

approaches. With SSL models we achieve signiﬁcant reduction

of the feature vectors size by two orders of magnitude, while

improving the performance by a factor ranging from 2 to 2.5

across the evaluation metrics, supported by visual assessment.

Furthermore, we showcase how adapting the reference SSL

architecture to domain-speciﬁc data is followed by a substantial

reduction in model complexity up to one order of magnitude,

without compromising, and in some cases, even improving the

clustering performance.

Index Terms—spectrum analysis clustering self-supervised ma-

chine learning

I. INTRODUCTION

The number and type of wireless devices connected to

the Internet is rapidly increasing with the current affordable

personal mobile and Internet of Things (IoT) devices, requiring

wireless networks to handle large number of connections and

high trafﬁc loads. As a reference, the requirement for the num-

ber of connected devices in the ﬁfth-generation (5G) networks

is one million devices per square kilometer. The existence of

such a number of devices requires complex wireless resource

management. Over time, several new approaches to wireless

resource sharing, including dynamic spectrum access [1],

licensed shared access [2] have been proposed. However,

additional technological components, such as spectrum usage

databases [3] and radio environment maps [4], had to be

developed to enable such sophisticated and dynamic spectrum

usage approaches. To be able to correctly inform on spectrum

usage, additional knowledge of other devices operating within

the range of a wireless device is critical for future smart usage

of the spectrum. In this respect, some of the recent efforts

were focused on detecting the modulation used [5], technology

used [6], anomalous activities [7], etc.

As also discussed in [8] and [9], signiﬁcant effort is still

being invested in the ﬁeld to develop accurate and scalable

deep learning (DL) algorithms able to accurately and au-

tomatically manage spectrum resource usage. With respect

to the learning approach, these techniques can be divided

into (1) supervised that require labels to be present for the

training data, and (2) unsupervised that do not assume any

such labels. Applications in wireless spectrum management

need to be aware of operating details (i.e., type of technology,

transmission parameters, etc.). Development of a DL-based

model to support such application typically requires labelled

data that is expensive to acquire as it requires complex wireless

and computing equipment [10], [11] or intense labelling efforts

by domain experts [12] that do not always lead to high quality

labels due to the nondeterministic nature of wireless operating

environments. Semi-supervised and active-learning emerged

as alternative techniques that have the advantage of using

a relatively small amount of labeled samples for achieving

performance that is comparable to the regular supervised

approach.

Given the advent of large datasets which are expensive

or practically impossible to label, self-supervised learning

(SSL) [13], as another intermediate learning approach, is

becoming an important alternative that is particularly suitable

to reduce the data labelling cost and leverage the unlabelled

data pool. SSL is a representation learning method where a

supervised task is created out of the unlabelled data. Using

an SSL approach, it is possible to create very similar groups

(i.e. clusters) from a large, unlabelled dataset and then label

each cluster. By labelling the learnt clusters, it is possible to

then use the model as a classiﬁer by assigning new, unseen

examples to those clusters and therefore label them as one

would do in a typical classiﬁcation task.

Developing an easy to use, automated and technology

agnostic way to explore spectrum activities and group similar

activities, eventually enabling automatic rather than manual

transmission identiﬁcation and cataloguing as currently done

for instance in the Signal Identiﬁcation Guide1, is still an open

research topic which motivated this investigation.

1https://www.sigidwiki.com/wiki/Signal Identiﬁcation Guide

arXiv:2210.02899v3 [cs.NI] 22 Aug 2024

Employing DL architectures based on convolutional neural

networks (CNN) allows for direct processing and classiﬁcation

of the raw In-phase and Quadrature (I/Q) time series [14].

While for modulation classiﬁcation [5] and device ﬁngerprint-

ing [15], direct processing on the raw I/Q time series data

is desirable, for radio access technology classiﬁcation it is

better to employ spectrograms due to lower complexity and

robustness towards low SNR conditions [6]. Additionally, the

image-like (2D matrices) format of the spectrograms provide

the possibility to utilize some state-of-the-art architectures

from the closely related machine vision domain, such as Vision

Transformers (ViT) [16] that recently emerged as competitors

to the CNN-based solutions. Inter domain adaptations, such as

in [17], could additionally stimulate the development of spec-

trum sensing technologies which are increasingly important

for the next generation radio networks.

In this paper, we investigate the suitability of SSL to

support automatic spectrum exploration on an example of an

unlicensed 868 MHz Short Range Device (SRD) band in an

urban environment using 15 days of spectrum sweeps collected

in the LOG-a-TEC2wireless testbed. Leveraging this data,

we propose an SSL architecture adapted for spectrum activity

identiﬁcation and clustering, in which segments of spectro-

grams containing signal activity are used to train the DL self-

supervised network and enable the discovery of the types of

transmissions available over the respective period of time. It

is based on machine vision and inspired by DeepCluster [18],

which is also used in this study as a reference SSL model.

In this architecture, we experiment with CNN-based and ViT-

based feature extraction and prove that such an architecture is

suitable for spectrogram analysis by learning spectrum features

and clustering spectrograms based on their content.

The main contributions of this work can be summarised as

follows:

•Adaptation of an SSL architecture to discover wireless

transmissions in real-world spectrogram data when no

prior knowledge (i.e. labels) is available, while also

achieving signiﬁcant reduction of the complexity of the

architecture with up to 10 times less trainable parameters

compared to the original implementation.

•Proposing dimensionality reduction of the embedding

space by principal component analysis (PCA) using a

threshold on the amount of explained variance ratio

(EVR), achieving features quality and clustering per-

formance improvement by a factor of 2-2.5 across the

selected evaluation metrics.

•Development of a methodology for quantitative and qual-

itative evaluation of the transmissions clustering from

raw spectrograms. The methodology consists of features

quality assessment in the embedded space and clustering

evaluation with selected metrics, veriﬁed by data-speciﬁc

visualizations.

•Experimental evaluation of the adapted SSL architecture

for two use cases, activity detection and ﬁne grain trans-

missions classiﬁcation.

2https://log-a-tec.eu/datasets.html

•Comparison of two feature extraction modules in the

adapted SSL architecture, one based on ViT, considered

as more powerful in RGB image classiﬁcation tasks, and

CNN, that proved more suitable for low-content data such

as spectrograms, taking into account the performance and

complexity trade-offs.

The rest of the paper is structured as follows. Section

II analyzes the related work, Section III introduces SSL

and baseline system architectures, Section IV elaborates on

the experimental methodology and Section V presents the

experimental results. Finally, Section VI concludes the paper.

II. RELATED WORK

In recent years, as in many other research areas the use of

DL models gained a lot of attention also in the development

of algorithms for processing spectrum data. Selected works

from the domain which are considered as most relevant and

closely related to this paper are listed in Table I, which

summarizes their main characteristics, and brieﬂy elaborate in

the following where they are grouped according to the adopted

learning approach.

A. Supervised learning

In the existing works, supervised DL-based models are

most widely represented and they achieve signiﬁcant increase

of performance when compared to more traditional ML ap-

proaches, as shown in [6], [14], [20].

In [6], the authors compare the performance and general-

ization ability of models that use manually extracted expert

features with models that use raw spectrum data. They are

solving the task of technology classiﬁcation on a dataset con-

taining transmissions of three different wireless technologies.

They prove that using CNN on raw I/Q data or spectrogram

images outperforms all other models in terms of accuracy, gen-

eralization ability to unseen datasets from different operating

environments, and robustness to different noise levels.

In [20], an application of CNN supervised learning for

device identiﬁcation, again using raw I/Q data, is proposed.

The dataset contains transmissions from ﬁve devices. Device

identiﬁcation is based on the CNN’s ability to learn various

device-speciﬁc impairments in the raw signals. SVM and lo-

gistic regression are used as reference models for performance

comparison and authors show that the proposed CNN model

signiﬁcantly outperforms the baseline algorithms for the posed

task.

In many cases, state-of-the-art accuracy is achieved by

adopting and modifying DL-based architectures that are al-

ready known and well established in other, closely related

signal processing ﬁelds, such as image and sound processing

[19], [7]. In [15], device classiﬁcation task using real-world

I/Q data of transmissions from a large population of nearly

10,000 devices is solved using a new neural network architec-

ture based on dilated causal convolutional layers. The design is

motivated by an existing audio signals processing architecture.

TABLE I: Related works with main characteristics

Publi-

cation

Problem type Architec-

ture

Data type Dataset Conti-nuous

sensing (yes/no)

Labels

(yes/no)

Approach Band

[5] Modulation classiﬁcation

(multiclass)

LSTM I/Q, PSD RadioML,

Electrosense

Yes Yes Supervised 174-230 MHz,

470-862MHz,

25-1300 MHz

[19] Device ﬁngerprinting (mul-

ticlass)

DCC I/Q DARPA,

Synthetic

/ Yes Supervised 2.4GHz, 5GHz,

978MHz,

1090MHz

[20] Device ﬁngerprinting (mul-

ticlass)

CNN I/Q Test-bed / Yes Supervised 5GHz

[6] Technology classiﬁcation

(multiclass)

DT, FNN,

CNN

RSSI, I/Q,

Spectro-

grams

Ghent, Dublin Yes Yes Supervised 5540MHz,

2412MHz,

806MHz

[14] Modulation classiﬁcation

(multiclass)

CNN I/Q Test-bed,

synthetic

/ Yes Supervised 900MHz

[17] Technology characteriza-

tion trough object detection

(multiclass)

YOLO CNN Spectro-

grams

Test-bed, Ghent Yes Yes Supervised 5540MHz,

2412MHz,

806MHz

[21] Device ﬁngerprinting with

clustering (multiclass)

CNN I/Q Test-bed / Yes Semi-supervised 0.25 - 1.25MHz,

1.67MHz,

2.5MHz

[15] Device ﬁngerprinting (mul-

ticlass)

DCC +

CNN

I/Q DARPA / Yes Supervised, Unsu-

pervised

2.4GHz, 5GHz,

978MHz,

1090MHz

[7] Anomaly detection

(binary)

PredNet-

autoencoder

Spectro-

grams

Synthetic / No Unsupervised* /

[22] Anomaly detection

(binary)

Supervised

CNN-based

PSD Synthetic,

HackRF SDR,

Electrosense

Yes Yes Unsupervised 10MHz-3GHz

[12] Transmissions detection

and classiﬁcation (binary)

Image pro-

cessing

Spectro-

grams

Log-A-Tec Yes Yes Unsupervised 868MHz

Ours Transmissions clustering

(multiclass)

Deep-

clustering

Spectro-

grams

Log-A-Tec Yes No Unsupervised

(Self-supervised)

868MHz

B. Semi-supervised and transfer learning

Although the supervised models have superior performance

judging by their accuracy, the necessity for large labeled train-

ing datasets as one of the major downsides of this approach

remains. One way of going around this problem is to train

feature extractor in unsupervised manner and then tune the

classiﬁcation on a small labelled dataset. This approach is

employed in [22] to solve the problem of anomaly detection

in wireless communications.

Another approach for the labeling problem is employing

transfer learning [19], [17]. In [17], object detection ”You

Only Look Once” (YOLO) model, pretrained on the ImageNet

dataset, is tuned to detect and classify different types of trans-

missions using spectrograms. It is shown that the proposed

model performs well in classifying interfering signals on sim-

ulated data with different signal-to-noise ratios and provides

additional information about the position of the transmission

events in the spectrum.

Although the semi-supervised and transfer learning are

proven to lower the amount of data that is required for the

training, labeled data is still needed for the tuning. Spectrum

data content is not as universal as the RGB images and it is

much more dependent on the operating environment regarding

the noise, signal strength, fading, multipath effects, etc., which

makes it necessary to provide labels for each speciﬁc radio

environment.

C. Unsupervised learning

Considering the availability of large amounts of unlabelled

spectrum data [11], [10], another approach to the labeling

problem with high potential, which we consider as under-

explored in the spectrum data processing domain, is using

unsupervised models. We believe that the development of

unsupervised approaches could more effectively solve the

labeling problems and provide highly automated architectures

that can be adapted to different operating environments with

minimum expert intervention.

There are existing efforts in this direction, but they only

consider the marginal cases of event detection (binary clas-

siﬁcation). A pipeline for automatic detection of wireless

transmissions using classic image processing techniques is

proposed in [12]. Another such example is the work in [23]

where auto-encoder is utilized as feature learner for anomaly

detection and shown to outperform the robust PCA.

Some recent works employ contrastive learning as an SSL

representation learning approach, for instance on I/Q samples

[24] and motion sensor data [25]. Although they show promis-

ing performance, this approach is more suitable for datasets

that have clear distinction between positive and negative

samples, while also requiring data-speciﬁc transformations for

training.

In our work, we align with the efforts of developing

completely unsupervised models that would provide feature

learning in an automated manner. We develop a self-supervised

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

1Self-supervisedLearningforClusteringofWirelessSpectrumActivityLjupchoMilosheski∗†,GregorCerar∗,BlaˇzBertalaniˇc∗,CarolinaFortuna∗,MihaelMohorˇciˇc∗†∗DepartmentofCommunicationTechnologies,JozefStefanInstitute,Jamova39,1000,Ljubljana,Slovenia†JozefStefanInternationalPostgraduateSchool,Jamova39,1000,L...

展开>> 收起<<

1 Self-supervised Learning for Clustering of Wireless Spectrum Activity.pdf

共15页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

1 Self-supervised Learning for Clustering of Wireless Spectrum Activity

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: