Meta Input How to Leverage Off-the-Shelf Deep Neural Networks Minsu Kim Youngjoon Yu Sungjune Park Yong Man Ro Image and Video Systems Lab KAIST South Korea

2025-05-02 0 0 442.6KB 9 页 10玖币

侵权投诉

Meta Input: How to Leverage Off-the-Shelf Deep Neural Networks

Minsu Kim*, Youngjoon Yu*, Sungjune Park*, Yong Man Ro

Image and Video Systems Lab, KAIST, South Korea

{ms.k, greatday, sungjune-p, ymro}@kaist.ac.kr

Abstract

These days, although deep neural networks (DNNs) have

achieved a noticeable progress in a wide range of research

area, it lacks the adaptability to be employed in the real-world

applications because of the environment discrepancy prob-

lem. Such a problem originates from the difference between

training and testing environments, and it is widely known

that it causes serious performance degradation, when a pre-

trained DNN model is applied to a new testing environment.

Therefore, in this paper, we introduce a novel approach that

allows end-users to exploit pretrained DNN models in their

own testing environment without modifying the models. To

this end, we present a meta input which is an additional in-

put transforming the distribution of testing data to be aligned

with that of training data. The proposed meta input can be

optimized with a small number of testing data only by con-

sidering the relation between testing input data and its output

prediction. Also, it does not require any knowledge of the

network’s internal architecture and modiﬁcation of its weight

parameters. Then, the obtained meta input is added to testing

data in order to shift the distribution of testing data to that

of originally used training data. As a result, end-users can

exploit well-trained models in their own testing environment

which can differ from the training environment. We validate

the effectiveness and versatility of the proposed meta input by

showing the robustness against the environment discrepancy

through the comprehensive experiments with various tasks.

Introduction

Recently, as the deep learning has achieved a great devel-

opment, deep neural networks (DNNs) have shown remark-

able performances in various research areas, such as com-

puter vision (He et al. 2016), natural language processing

(Vaswani et al. 2017), and speech processing (Amodei et al.

2016). Nevertheless, there exists one signiﬁcant problem to

be solved in utilizing DNNs robustly in the real-world, that

is, the environment discrepancy problem. The environment

discrepancy problem occurs when the training data distri-

bution and testing data distribution are mismatched, and it

results in the serious performance degradations of DNNs

(Klejch et al. 2019; Touvron et al. 2019; Teney et al. 2020).

Therefore, although end-users want to exploit well-trained

DNNs in their own testing environment, they would fail to

*These authors contributed equally.

experience the powerfulness of DNNs because of the afore-

mentioned problem. For example, as described in Fig. 1(a),

an end-user tries to adopt an object detection model which is

trained with clean weather data, and of course, it detects ob-

jects successfully under thte same clean weather condition.

However, as shown in Fig. 2(b), when the user wants to de-

tect objects under adverse weather condition, the detection

model would fail to conduct the robust object detection, be-

cause the testing environment (i.e., adverse weather) differs

from the training environment (i.e., clean weather) (Huang,

Hoang, and Le 2022; Sakaridis, Dai, and Van Gool 2018).

Therefore, end-users are recommended to ﬁnd and exploit

the DNN models well-trained on the training data that is

consistent with their own testing environment. One possi-

ble approach to alleviate such a problem is Domain Adap-

tation (DA) which aims to reduce the domain gap between

the source and target domains by learning the domain in-

variant representations (Xiao and Zhang 2021; Saito et al.

2018; Bousmalis et al. 2017; Ganin and Lempitsky 2015;

Long et al. 2017). However, such DA methods are usually

required to know the internal architecture of the network and

have an access to both source and target data simultaneously

for learning the domain invariant features. Therefore, it is

time-consuming and difﬁcult for end-users to understand the

behavior of the network and use both kinds of data.

In this paper, we focus on how to make end-users en-

joy the beneﬁcial performances of well-trained DNN models

even with different testing data distribution. To this end, we

introduce a method that end-users can adapt the pretrained

model into their testing environment without any knowledge

of model architecture or ﬁnetuning the model, by using only

a small number of data in testing environment. Motivated

by the recent success in the input-level transformation to

convert the originally learned task to another task (Elsayed,

Goodfellow, and Sohl-Dickstein 2018), instead of modifying

the weight parameters of the pretrained models, we propose

to use an additional input, called meta input, to match the

distributions of testing data with that of training data. Specif-

ically, we suppose that an end-user wants to adopt a pre-

trained model under different testing environment with a few

labeled/unlabeled testing data only, while the user cannot

have an access to the training data which is used to pretrain

the model. Then, the proposed meta input can be optimized

to transform the testing data distribution to be aligned with

arXiv:2210.13186v1 [cs.LG] 21 Oct 2022

Object

Detector

Testing Environment : Clean Weather Condition

Testing Data Robust Detection

(a)

Testing Environment : Adverse Weather Condition

Object

Detector

Testing Data Detection Failure

(b)

Testing Environment : Adverse Weather Condition

Object

Detector

Testing Data Robust Detection

(c)

Meta Input

Figure 1: (a) describes that the object detector which is trained on clean weather data can perform the object detection suc-

cessfully. However, (b) shows that, when end-users exploit the detector under the adverse weather condition, it usually fails

the detection because of the mismatch between the training and testing environments. To alleviate the problem, we introduce a

meta input which is embedded into testing data and transforms the distribution of testing data (i.e., adverse weather) to that of

training data (i.e., clean weather). So that, the detector becomes to conduct the object detection properly.

the training data distribution where the pretrained model op-

erates properly. After that, the meta input can be embedded

into the testing data to make the pretrained model perform

well under different testing environment. For example, as

shown in Fig. 1(c), the meta input is embedded into the test-

ing data, so that the pretrained detection model conducts the

robust object detection even under adverse weather condi-

tion without modifying its weight parameters.

The proposed meta input can be optimized simply with

any gradient-based training algorithm by using a few la-

beled, or unlabeled, data of testing environment. With the

meta input, the learned knowledge of pretrained DNN mod-

els can be extended to diverse testing environments without

knowing the network architecture and modifying its weight

parameters. Therefore, end-users can enjoy the powerfulness

of off-the-shelf DNNs on their own testing environment. We

verify both effectiveness and practicality of the proposed

meta input in the real-world through the extensive experi-

ments in the three tasks, image classiﬁcation, object detec-

tion, and visual speech recognition.

Our contributions can be summarized as follows:

• Since the proposed meta input can match the distribution

of testing data with that of training data, the knowledge

the pretrained DNN models already learned can be uti-

lized properly even under different environments.

• Different from the existing DA methods, the proposed

method does not require any knowledge of the model

architecture, modiﬁcation of its weight parameters and

training data (which is used for pretraining the model),

and it only needs a small number of testing data.

• The effectiveness and versatility of the proposed meta in-

put are corroborated by the comprehensive experiments

on three practical tasks, image classiﬁcation, object de-

tection, and visual speech recognition.

Related Work

Domain Adaptation

Deep Neural Networks (DNNs) have been widely adopted

to extract the generalized feature representation of the data.

To train such a generalized DNNs, it assumes that both

training and testing data are originated from the same dis-

tribution and share some similar joint probability distribu-

tion. In the real-world scenario, however, this constraint is

easily violated, because each training and test data can be

drawn from different distributions. To tackle the aforemen-

tioned problems, researchers have devoted their efforts on

a research ﬁeld called Domain Adaptation (DA) (Wang and

Deng 2018; Kang et al. 2019). DA is a technique that enables

DNNs learned with sufﬁcient label and data size (i.e., source

domain) to perform and generalize well on data sampled

from different distributions (i.e., target domain). DA can

be categorized into discrepancy-based methods (Long et al.

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

MetaInput:HowtoLeverageOff-the-ShelfDeepNeuralNetworksMinsuKim*,YoungjoonYu*,SungjunePark*,YongManRoImageandVideoSystemsLab,KAIST,SouthKoreafms.k,greatday,sungjune-p,ymrog@kaist.ac.krAbstractThesedays,althoughdeepneuralnetworks(DNNs)haveachievedanoticeableprogressinawiderangeofresearcharea,itlacksth...

展开>> 收起<<

Meta Input How to Leverage Off-the-Shelf Deep Neural Networks Minsu Kim Youngjoon Yu Sungjune Park Yong Man Ro Image and Video Systems Lab KAIST South Korea.pdf

共9页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Meta Input How to Leverage Off-the-Shelf Deep Neural Networks Minsu Kim Youngjoon Yu Sungjune Park Yong Man Ro Image and Video Systems Lab KAIST South Korea

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: