1
Signal Detection in MIMO Systems with Hardware
Imperfections: Message Passing on Neural
Networks
Dawei Gao, Qinghua Guo, Guisheng Liao, Yonina C. Eldar, Fellow, IEEE, Yonghui Li, Fellow, IEEE, Yanguang
Yu, and Branka Vucetic, Fellow, IEEE
Abstract—In this paper, we investigate signal detection in
multiple-input-multiple-output (MIMO) communication systems
with hardware impairments, such as power amplifier nonlinearity
and in-phase/quadrature imbalance. To deal with the complex
combined effects of hardware imperfections, neural network (NN)
techniques, in particular deep neural networks (DNNs), have
been studied to directly compensate for the impact of hardware
impairments. However, it is difficult to train a DNN with limited
pilot signals, hindering its practical applications. In this work,
we investigate how to achieve efficient Bayesian signal detection
in MIMO systems with hardware imperfections. Characterizing
combined hardware imperfections often leads to complicated
signal models, making Bayesian signal detection challenging. To
address this issue, we first train an NN to ‘model’ the MIMO
system with hardware imperfections and then perform Bayesian
inference based on the trained NN. Modelling the MIMO system
with NN enables the design of NN architectures based on the
signal flow of the MIMO system, minimizing the number of NN
layers and parameters, which is crucial to achieving efficient
training with limited pilot signals. We then represent the trained
NN with a factor graph, and design an efficient message passing
based Bayesian signal detector, leveraging the unitary approxi-
mate message passing (UAMP) algorithm. The implementation
of a turbo receiver with the proposed Bayesian detector is also
investigated. Extensive simulation results demonstrate that the
proposed technique delivers remarkably better performance than
state-of-the-art methods.
Index Terms—Hardware imperfections, I/Q Imbalance, power
amplifier nonlinearity, multiple-input-multiple-output (MIMO),
neural networks (NNs), factor graphs, approximate message
passing (AMP), Bayesian inference.
I. INTRODUCTION
WE consider signal detection for multiple-input multi-
output (MIMO) communications in the presence of
hardware impairments, which arise, e.g., in millimeter wave
(mm-wave) communications, where mm-wave front ends suf-
fer from significant hardware imperfections, compromising
Corresponding to Qinghua Guo (qguo@uow.edu.au).
Dawei Gao and Guisheng Liao are with the Hangzhou Institute of Technol-
ogy, Xidian University, Hangzhou 311200, China and also with the National
Laboratory of Radar Signal Processing, Xidian University, Xi’an 710071,
China (e-mail: gaodawei@xidian.edu.cn; liaogs@xidian.edu.cn).
Qinghua Guo and Yanguang Yu are with the School of Electrical, Computer
and Telecommunications Engineering, University of Wollongong, NSW 2522,
Australia (e-mail: qguo@uow.edu.au; yanguang@uow.edu.au).
Yonina C. Eldar is with the Faculty of Math and CS, Weizmann Institute
of Science, Rehovot, 7610001, Israel (email: yonina.eldar@weizmann.ac.il).
Yonghui Li and Brank Vucetic are with the School of Electrical and
Information Engineering, University of Sydney, Sydney, NSW 2006, Australia
(e-mail: yonghui.li@ sydney.edu.au; branka.vucetic@sydney.edu.au).
signal transmission quality and degrading system performance
[1]–[3]. A pronounced impairment is in-phase/quadrature (I/Q)
imbalance, i.e., the mismatch of amplitude, phase and fre-
quency response between the I and Q branches, which impairs
their orthogonality [4]. Power amplifier (PA) nonlinearity leads
to nonlinear distortions to transmitted signals, which cannot be
overlooked, especially in mm-wave communications [2]. The
hardware imperfections need to be handled properly to avoid
inducing significant system performance loss.
Many techniques have been considered to mitigate the
impact of hardware imperfections. To handle PA nonlinearity,
Volterra series based techniques were proposed for nonlinear-
ity compensation at either transmitter or receiver [5], [6]. How-
ever, these techniques often need to determine a large number
of Volterra series coefficients, which is a difficult task. To
address this, some simplified methods such as those based on
memory polynomials [7], Hammerstein model [8] and Wiener
model [9] were proposed [7]. Addressing I/Q imbalance has
also attracted much attention [10]–[13]. In [12], a dual-
input nonlinear model based on a real-valued Volterra series
was proposed to model the I/Q imbalance, and its inverse
model was employed at the transmitter to pre-compensate
the I/Q imbalance. In [13], a single-user point-to-point mm-
wave hybrid beamforming system with I/Q imbalance at the
transmitter and its pre-compensation were considered. The
pre-compensation technique [13] assumes the availability of
instantaneous channel state information at the transmitter,
which can be difficult to achieve in practical scenarios. With
higher orders, polynomial-based techniques have potential to
handle severer nonlinear distortions, which, however, are more
prone to numerical instability in determining their coefficients
[14]–[16]. We also note that, most of the polynomial-based
algorithms in the literature deal with a single type of hardware
imperfections, i.e., either PA nonlinearity or I/Q imbalance.
However, hardware imperfections may occur at the same time,
leading to combined effects.
Neural networks (NNs) have recently emerged as a promis-
ing technique to deal with the nonlinear effects in communica-
tion systems [17]–[19]. In [20], a real-valued time-delay neural
network (RVTDNN) was proposed to model PA behaviors.
Various variants of RVTDNN were proposed [21], [22] to
address the combined effects of hardware impairments. In [21],
high-order signal components are applied to the RVTDNN to
pre-compensate both the PA nonlinearity and I/Q imbalance.
In [22], a deep NN (DNN) based technique was proposed to
arXiv:2210.03911v1 [eess.SP] 8 Oct 2022