Learning Failure-Inducing Models for Testing Software-Defined Networks

2025-05-02 0 0 959.04KB 26 页 10玖币

侵权投诉

Learning Failure-Inducing Models for Testing

Soware-Defined Networks

RAPHAËL OLLANDO, University of Luxembourg, Luxembourg

SEUNG YEOB SHIN, University of Luxembourg, Luxembourg

LIONEL C. BRIAND, Lero centre, University of Limerick, Ireland and University of Ottawa, Canada

Software-dened networks (SDN) enable exible and eective communication systems that are managed by

centralized software controllers. However, such a controller can undermine the underlying communication

network of an SDN-based system and thus must be carefully tested. When an SDN-based system fails, in

order to address such a failure, engineers need to precisely understand the conditions under which it occurs.

In this article, we introduce a machine learning-guided fuzzing method, named FuzzSDN, aiming at both

(1) generating eective test data leading to failures in SDN-based systems and (2) learning accurate failure-

inducing models that characterize conditions under which such system fails. To our knowledge, no existing

work simultaneously addresses these two objectives for SDNs. We evaluate FuzzSDN by applying it to systems

controlled by two open-source SDN controllers. Further, we compare FuzzSDN with two state-of-the-art

methods for fuzzing SDNs and two baselines for learning failure-inducing models. Our results show that

(1) compared to the state-of-the-art methods, FuzzSDN generates at least 12 times more failures, within the

same time budget, with a controller that is fairly robust to fuzzing and (2) our failure-inducing models have,

on average, a precision of 98% and a recall of 86%, signicantly outperforming the baselines.

CCS Concepts: •Software and its engineering

→

Software testing and debugging;•Networks

→

Programmable networks.

Additional Key Words and Phrases: Software-Dened Networks, Software Testing, Fuzzing, Machine Learning

ACM Reference Format:

Raphaël Ollando, Seung Yeob Shin, and Lionel C. Briand. 2024. Learning Failure-Inducing Models for Testing

Software-Dened Networks. 1, 1 (January 2024), 26 pages. https://doi.org/10.1145/nnnnnnn.nnnnnnn

1 INTRODUCTION

Software-dened networks (SDN) [

] have emerged to enable programmable networks that allow

system operators to manage their systems in a exible and ecient way. SDNs have been widely

deployed in many application domains, such as data centers [

], the Internet of Things [

and satellite communications [

]. The main idea behind SDNs is to transfer the control of

networks from localized, xed-behavior controllers distributed over a set of network switches

(in traditional networks) to a logically centralized and programmable software controller. With

complex software being an integral part of SDNs, developing SDN-based systems (SDN-systems),

e.g., data centers, entails interdisciplinary considerations, including software engineering.

Authors’ addresses: Raphaël Ollando, raphael.ollando@uni.lu, University of Luxembourg, 29 Avenue John F. Kennedy,

Luxembourg, 1859, Luxembourg; Seung Yeob Shin, seungyeob.shin@uni.lu, University of Luxembourg, 29 Avenue John F.

Kennedy, Luxembourg, 1859, Luxembourg; Lionel C. Briand, lionel.briand@lero.ie, Lero centre, University of Limerick,

Tierney building, Limerick, V94 NYD3, Ireland and University of Ottawa, 800 King Edward Avenue, Ottawa, ON K1N 6N5,

Canada.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee

provided that copies are not made or distributed for prot or commercial advantage and that copies bear this notice and

the full citation on the rst page. Copyrights for components of this work owned by others than ACM must be honored.

Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires

prior specic permission and/or a fee. Request permissions from permissions@acm.org.

XXXX-XXXX/2024/1-ART $15.00

https://doi.org/10.1145/nnnnnnn.nnnnnnn

, Vol. 1, No. 1, Article . Publication date: January 2024.

arXiv:2210.15469v3 [cs.SE] 8 Jan 2024

2 Raphaël Ollando, Seung Yeob Shin, and Lionel C. Briand

In the context of developing SDN-systems, software testing becomes even more important and

challenging when compared to what is required in traditional networks that provide static and

predictable operations. In particular, even though the centralized controller in an SDN-system

enables exible and ecient services, it can undermine the entire communication network it man-

ages. A software controller presents new attack surfaces that allow malicious users to manipulate

the systems [

]. For example, if malicious users intercept and poison communication in the

system (using ARP spoong [

]), such attacks broadly impact the system due to its centralized

control. Furthermore, the centralized controller interacts with diverse kinds of components such as

applications and network switches, which are typically developed by dierent vendors. Hence, the

controller is prone to receiving unexpected inputs provided by applications, switches, or malicious

users, which may cause system failures, e.g., communication breakdown.

To test an SDN controller, engineers need rst to explore its possible input space, which is

very large. A controller takes as input a stream of control messages which are encoded according

to an SDN communication protocol (e.g., OpenFlow [

]). For example, if a control message is

encoded with OpenFlow, it can have 2

2040

distinct values [

]. Second, engineers need to under-

stand the characteristics of test data, i.e., control messages, that cause system failures. However,

manually inspecting test data that cause failures is time-consuming and error-prone. Furthermore,

misunderstanding such causes typically leads to unreliable xes.

There are a number of prior research strands that aim at testing SDN-systems [

]. Most of them come from the network research eld and focus on security testing

relying on domain knowledge, e.g., known attack scenarios [

]. The most pertinent research

works applied fuzzing techniques to dierent components of SDN-systems. For example, RE-

CHECKER [

] fuzzes RESTful services provided by SDN controllers. SwitchV [

] relies on fuzzing

and symbolic execution to test SDN switches. BEADS [

] tests SDN controllers by being aware

of the OpenFlow specication. However, none of these fuzzing techniques employ interpretable

machine learning techniques to guide their fuzzing process and to provide models that characterize

failure-inducing conditions. Even though the software engineering community has introduced

numerous testing methods, testing SDN-systems has gained little attention. The most pertinent

research lines have proposed techniques for learning-based fuzzing [

] and abstracting

failure-inducing inputs [

] to eciently explore the input space and characterize eective

test data that cause system failures. Learn@Fuzz [

] employs neural-network-based learning

methods for building a model of PDF objects for grammar-based fuzzing. A prior learning-guided

fuzzing technique, named smart fuzzing [

], relies on deep learning techniques to test systems

controlled by programmable logic controllers (PLC). SeqFuzzer [

] uses deep learning techniques

to infer communication protocols underlying PLC systems and generate fuzzed messages. However,

deep learning techniques are not suitable to characterize failure-inducing test data as they do not

provide interpretable models. Furthermore, these techniques do not account for the specicities of

SDNs, such as SDN architecture and communication protocol. Existing work on abstracting failure-

inducing inputs [

] targets software programs that take as input strings such as command-line

utilities (e.g., nd and grep), which are signicantly dierent from SDN-systems. In summary,

no existing work simultaneously tackles the problem of eciently exploring the input space and

accurately characterizing failure-inducing test data while accounting for the specicities of SDNs.

Contributions. In this article, we propose FuzzSDN, a machine learning-guided

Fuzz

ing method

for testing

SDN

-systems. In particular, FuzzSDN targets software controllers deployed in SDN-

systems. FuzzSDN relies on fuzzing guided by machine learning (ML) to both (1) eciently explore

the test input space of an SDN-system’s controller (generate test data leading to system failures) and

(2) learn failure-inducing models that characterize input conditions under which the system fails.

This is done in a synergistic manner where models guide test generation and the latter also aims at

, Vol. 1, No. 1, Article . Publication date: January 2024.

Learning Failure-Inducing Models for Testing Soware-Defined Networks 3

controller

switch1switch2switch3

host1host2host3host4

: control channel : data channel

control layer

infrastructure

layer

Fig. 1. An SDN topology example containing one controller, three switches, and four hosts.

improving the models. A failure-inducing model is practically useful [

] for the following reasons:

(1) It facilitates the diagnosis of system failures. FuzzSDN provides engineers with an interpretable

model specifying how likely are failures to occur, e.g., the system fails when a control message is

encoded using OpenFlow V1.0 and contains IP packets, thus providing concrete conditions under

which a system will probably fail. Such conditions are much easier to analyze than a large set of

individual failures. (2) A failure-inducing model enables engineers to validate their xes. Engineers

can x and test their code against the generated test data set. A failure-inducing model can also

be used as a test data generator to reproduce the system failures captured in the model. Hence,

engineers can better validate their xes using an extended test data set.

We evaluated FuzzSDN by applying it to several systems controlled by well-known open-source

SDN controllers: ONOS [

] and RYU [

]. In addition, we compared FuzzSDN with two state-of-

the-art methods (i.e., DELTA [

] and BEADS [

]) that generate test data for SDN contollers and

two baselines that learn failure-inducing models. As baselines, we extended DELTA and BEADS

to produce failure-inducing models, since they were not originally designed for that purpose but

were nevertheless our best options. Our experiment results show that, compared to state-of-the-art

methods, FuzzSDN generates at least 12 times more failing control messages, within the same

time budget, with a controller that is fairly robust to fuzzing. FuzzSDN also produces accurate

failure-inducing models with, on average, a precision of 98% and a recall of 86%, which signicantly

outperform models inferred by the two baselines. Furthermore, FuzzSDN produces failure-inducing

conditions that are consistent with those reported in the literature [

], indicating that FuzzSDN is

a promising solution for automatically characterizing failure-inducing conditions and thus reducing

the eort needed for manually analyzing test results. Last, FuzzSDN is applicable to systems with

large networks as its performance does not depend on network size. Our detailed evaluation results

and the FuzzSDN tool are available online [44].

Organization. The rest of this article is organized as follows: Section 2 introduces the background

and denes the specic problem of learning failure-inducing models for testing SDN-systems.

Section 3 describes FuzzSDN. Section 4 evaluates FuzzSDN in a large empirical study. Section 5

compares FuzzSDN with related work. Section 6 concludes this article.

2 BACKGROUND AND PROBLEM DESCRIPTION

In this section, we describe the fundamental concepts of an SDN-system. We then discuss the

problem of identifying input conditions under which the system under test fails.

Controller. In general, an SDN-system is composed of two layers: infrastructure and control

layers. The infrastructure layer contains physical components, such as switches and links, that

build a physical network transferring data ows such as audio and video data streams. The control

, Vol. 1, No. 1, Article . Publication date: January 2024.

4 Raphaël Ollando, Seung Yeob Shin, and Lionel C. Briand

layer is a software component developed by software engineers to meet the system’s requirements.

Specically, the software controller manages the physical components in the infrastructure layer

to implement, for example, system-specic data forwarding, security management, and failure

recovery algorithms. Figure 1 shows an SDN topology example, containing a controller in the control

layer along with three switches and four hosts (such as servers and clients) in the infrastructure layer.

Note that this controller is the target of our testing, which is not present in traditional networking

systems. In this article, we simplied the SDN layered architecture [

] to clearly explain our

contributions by abstracting out network-specic details, e.g., SDN southbound interfaces, that are

not important to present this work.

In traditional networking systems, controllers are typically embedded in the physical devices,

operating based on their local congurations. This decentralized control approach contrasts with the

centralized control of SDN-systems. The SDN architecture centralizes decision-making, aiming for

enhanced eciency and dynamic network management. However, this can also lead to SDN-specic

failures caused by such central point of control.

Message. The SDN controller and switches in a network communicate by exchanging control

messages. A control message is encoded as a sequence of values following a specic communication

protocol. For instance, OpenFlow [

] is a de facto standard communications protocol used in

many SDN-systems [

], enabling communication between the control and infrastructure layers.

To exercise the behavior of the controller under test, therefore, testing explores the space of possible

control messages.

In traditional networking systems, communication between devices usually employs well-

established transport protocols, such as UDP [

] and TCP [

], for data transmission. For exchang-

ing routing and state information, routers and switches use standard routing protocols, such as

OSPF [

] and BGP [

]. Since these protocols were designed to serve specic purposes in the

context of traditional static networks, the messages they encode are more similar and simpler than

those used in SDN. In contrast, SDN messages, such as those in OpenFlow, carry a variety of data

including ow setups, modications, and statistics, allowing for more exibility in network services.

However, this also entails a wider range of potential failures that should be accounted for during

testing.

Failure. Like other software components, SDN controllers may have faults that can lead to ser-

vice failures perceivable by users. These failures can manifest in various forms in the context of

SDN-systems. Specically, previous studies on SDN testing [

] have investigated

the following failures relevant to SDN controllers: (1) Controller-switch disconnection. When

communications between the controller and the switches are unexpectedly disconnected, the

system obviously cannot operate as intended. For example, if a switch does not receive timely

commands from the controller due to this disconnection, the switch relies on outdated forwarding

rules installed earlier, possibly leading to dropping data ows entirely. (2) SDN operation stall.

This failure refers to situations where the required execution of an SDN operation is unexpectedly

prevented or delayed. For example, an SDN controller might stall the installation of forwarding

rules, aecting the construction of the communication path in the SDN. (3) Incorrect understanding

of network status. SDN controllers rely on a centralized view of the network to make control

decisions. If this centralized view becomes inconsistent with the actual network, regardless of the

reasons, e.g., receiving outdated or incorrect data from the network switches, the controller may

fail to make decisions that reect the network’s actual conditions. For example, if a link between

two switches is broken, but the controller’s view still considers it operational, the controller fails to

instruct the switches to reroute data ows around the broken link, resulting in dropped data ows.

(4) Overutilization of resources. SDN controllers may overutilize their processing and memory

units due to various reasons, such as handling an unexpectedly high volume of requests. For

, Vol. 1, No. 1, Article . Publication date: January 2024.

Learning Failure-Inducing Models for Testing Soware-Defined Networks 5

FuzzSDN

SDN-system

test

procedure

failure

detection

mechanism

test

data

failure-

inducing

model

fuzzing learning

planning

Fig. 2. An overview of our ML-guided Fuzzing method for testing SDN-systems (FuzzSDN).

example, such a failure caused by a distributed denial-of-service (DDoS) attack [

] targeting the

SDN controller can lead to slower response times and potential service outages. We note that, in

addition to failures specic to SDN controllers, failures observed in traditional networks, such as

communication breakdowns among hosts (e.g., data servers and clients) and network performance

degradation, are also relevant in SDN-systems as its infrastructure layer usually employs traditional

network protocols, such as TCP, UDP, and IP.

In traditional networking systems, failures tend to be localized. If a router or switch fails, only

its directly connected neighbours (i.e., localized segments of the network) are aected. Similarly, if

a server or client fails, only the applications and users directly reliant on that specic device are

impacted. In contrast, SDN-systems have a centralized failure point in the controller. If the controller

crashes or loses connection with the switches, the entire network can be aected. This central

point of potential failure makes rigorous testing essential to ensure robustness and reliability.

Problem. When developing and operating an SDN-system, engineers must handle system failures

that are triggered by unexpected control messages. In particular, engineers need to ensure that the

system behaves in an acceptable way in the presence of failures. In an SDN-system, its controller is

prone to receiving unexpected control messages from switches in the system [

]. For example,

network switches, which are typically developed by dierent vendors, may send control messages

that fall outside the scope of the controller’s expectations. Such unexpected messages can also

be sent by the switches due to various reasons such as malfunctions and bugs in the switches, as

well as inconsistent implementations of a communication protocol between the controller and

switches [

]. Furthermore, prior security assessments of SDNs have found several attack surfaces,

leading to vulnerable applications and communications protocols that enable malicious actors to

send manipulated messages over an SDN [29, 35].

When a failure occurs in an SDN-system, engineers need to determine the conditions under

which such failures occur. These conditions dene a set of control messages that cause the failure.

Identifying such conditions in a precise and interpretable form is in practice useful, as it enables

engineers to diagnose the failure with a clear understanding of the conditions that induce it. In

addition, engineers can produce an extended set of control messages by utilizing the identied

conditions to test the system after making changes to address the failure. In general, any x should

properly address other control messages that induce the same failure. Our work aims to both

eectively test an SDN-system’s controller by identifying control messages that lead to system

failures, and then automatically identify an accurate failure-inducing model that characterizes

conditions under which the SDN-system fails. Such conditions dene a set of failure-inducing

control messages.

, Vol. 1, No. 1, Article . Publication date: January 2024.

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

LearningFailure-InducingModelsforTestingSoftware-DefinedNetworksRAPHAËLOLLANDO,UniversityofLuxembourg,LuxembourgSEUNGYEOBSHIN,UniversityofLuxembourg,LuxembourgLIONELC.BRIAND,Lerocentre,UniversityofLimerick,IrelandandUniversityofOttawa,CanadaSoftware-definednetworks(SDN)enableflexibleandeffectivecomm...

展开>> 收起<<

Learning Failure-Inducing Models for Testing Software-Defined Networks.pdf

共26页,预览5页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Learning Failure-Inducing Models for Testing Software-Defined Networks

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: