Grow and Merge A Uniﬁed Framework for Continuous Categories Discovery Xinwei Zhang1 Jianwen Jiang2 Yutong Feng2 Zhi-Fan Wu2 Xibin Zhao1y

2025-05-06 0 0 1.83MB 13 页 10玖币

侵权投诉

Grow and Merge: A Uniﬁed Framework for

Continuous Categories Discovery

Xinwei Zhang1∗

, Jianwen Jiang2∗, Yutong Feng2, Zhi-Fan Wu2, Xibin Zhao1†,

Hai Wan1, Mingqian Tang2, Rong Jin2, Yue Gao1,3†

1BNRist, KLISS, School of Software, Tsinghua University

2Alibaba Group

3THUIBCS, BLBCI, Tsinghua University

xinwei-z21@mails.tsinghua.edu.cn

{jianwen.jjw,fengyutong.fyt,wuzhifan.wzf,mingqian.tmq,jinrong.jr}

@alibaba-inc.com, {zxb,wanhai,gaoyue}@tsinghua.edu.cn

Abstract

Although a number of studies are devoted to novel category discovery, most of

them assume a static setting where both labeled and unlabeled data are given at

once for ﬁnding new categories. In this work, we focus on the application scenarios

where unlabeled data are continuously fed into the category discovery system.

We refer to it as the

Continuous Category Discovery

(

CCD

) problem, which

is signiﬁcantly more challenging than the static setting. A common challenge

faced by novel category discovery is that different sets of features are needed for

classiﬁcation and category discovery: class discriminative features are preferred for

classiﬁcation, while rich and diverse features are more suitable for new category

mining. This challenge becomes more severe for dynamic setting as the system

is asked to deliver good performance for known classes over time, and at the

same time continuously discover new classes from unlabeled data. To address

this challenge, we develop a framework of

Grow and Merge

(

) that works by

alternating between a growing phase and a merging phase: in the growing phase, it

increases the diversity of features through a continuous self-supervised learning for

effective category mining, and in the merging phase, it merges the grown model

with a static one to ensure satisfying performance for known classes. Our extensive

studies verify that the proposed GM framework is signiﬁcantly more effective than

the state-of-the-art approaches for continuous category discovery.

1 Introduction

Human beings are good at grouping objects into category through clustering, and the deﬁnition of

categories are continuously expanding and updated over time. Recent developments of intelligent

visual systems can not only distinguish pre-deﬁned categories [

–

], but also discover new categories

from unlabeled data, a task that is known as novel category discovery [6–8].

Existing works on novel category discovery are limited to the static setting, where both labeled

data (by known classes) and unlabeled data (with potential unknown categories) are given at once.

In contrast, for real-world applications, unlabeled data are continuously fed into the system for

discovering new categories, making it a signiﬁcantly more challenging problem. Besides, current

studies for novel category discovery often assume that all the unlabeled data belong to the unknown

∗Equal contributions.

†Corresponding authors.

Preprint. Under review.

arXiv:2210.04174v1 [cs.LG] 9 Oct 2022

new categories, which is generally not true in real applications. In this work, we examine the dynamic

setup of novel category discovery where the system was initially given a set of data labeled by known

classes, and unlabeled data are continuously streamed into the system for discovering new classes.

The system is requested to consistently yield satisfying performance for known classes, and at the

same time, dynamically discover new categories from the streaming unlabeled data. We refer to it as

Continuous Category Discovery, or CCD for short.

We illustrate the process of CCD in Figure 1. It is comprised of two main stages: the initial stage

where a classiﬁcation model is trained by a set of labeled examples, and the continuous category

discovery stage where new categories are continuously discovered from a stream of unlabeled

data belonging to both known and unknown classes. A intuitive approach to address the dynamic

nature of CCD is to combine the existing methods for open-set recognition [

–

], novel category

discovery [

], and incremental learning [

]. This is however insufﬁcient because our

learning system has to accomplish two tasks at the same time, i.e., accurately classify instances into

the known classes, and discover new categories from an unlabeled data stream. It turns out that these

two task models usually produce different types of features: discriminative features on known classes

are preferred by classiﬁcation model, while rich and diverse features are critical for identifying new

classes, as illustrated in Figure 2. A simple combination of novel category discovery and incremental

learning will fail to address the trade-off consistently over time, which is further veriﬁed by our

empirical studies.

Unlabeled set 1 Unlabeled set t

…

Continuous Category Discovery Stage

…

Labeled set

Dolphin

Fox DolphinFox Novel Category 1 Dolphin

Fox Novel Category 1 Novel Category n

Initial Stage

Time-step 1 Time-step t Time

…

Continuous

Datasets

Classiﬁer

…

Known class

Unknown class

Figure 1:

Overview of the Continuous Category Discovery (CCD).

The continuous data stream is

mixed with unlabeled samples from both known and novel categories. CCD requires to distinguish

known categories, discover novel categories and merge the discovered categories into known set.

To address the challenge of continuous category discovery, we propose a framework of

Grow

and

Merge

, or

for short. After pre-training a static model

over the labeled data, we will update

model

with respect to unlabeled data stream by alternating between the growing phase and the

merging phase: in the growing phase, we will increase the diversity of features by continuously

training our model over received unlabeled data through a combination of supervised and self-

supervised learning; in the merging phase, we will merge the grown model with the static one by

taking a weighted combination of both models. By alternating between the growing and merging

phases, we are able to maintain a good performance for known classes, and at same time, the power

of discovering new categories. This is clearly visualized in Figure 2, where the ﬁrst two panels show

that existing approaches can do well on one of the two tasks but not both, and last panel shows that

features learned by the proposed GM framework works well for both tasks.

Finally, one of the common issues with continuous training is catastrophic forgetting. To alleviating

the forgetting effect as we are growing the number of categories over time, we maintain a small

set of labeled samples from known categories and pseudo-labeled samples from novel categories.

These selected examples are used in the growing phase to expand feature diversity for effective

category discovery. Extensive experimental results show that our proposed method consistently

shows satisfying performance under multiple practical scenarios compared with existing methods.

The main contributions of this paper are summarized as follows:

•

We study a new problem named continuous category discovery, or CCD, which better

reﬂects the challenge of category discovery in the wild. It needs to simultaneously maintain

a good performance for known categories and the ability of discovering novel categories.

•

We propose a framework of grow and merge, or GM, for CCD, that is able to resolve the

conﬂicts between the classiﬁcation task and the task of discovering new categories.

Figure 2: Features visualization of model A trained on known categories, model B trained for novel

category discovery (based on model A) and the proposed GM model.

•

We conduct experiments under four different settings to fully investigate the scenarios of

novel category discovery in the wild. The proposed method shows less forgetting of known

categories and better performance for category discovery compared to existing methods.

2 Problem Deﬁnition

In this section, we formulate the problem of Continuous Category Discovery (CCD). There are two

main stages of the CCD problem, i.e.,

Initial Stage

and

Continuous Category Discovery Stage

The settings of each stage are introduced, together with the evaluation metrics of CCD problem.

2.1 Setting of Continuous Category Discovery

During the initial stage, a labeled training dataset

train ={(x0

i, y0

i)}N0

i=1

is provided to train the

model on the initial known category set

C0={1,2, ..., K0}

, where

is the initial training data, and

i∈ C0

is the corresponding label. The model is expected to classify categories in

, and learn

meaningful representation from the labels’ semantic information.

During the continuous category discovery stage, a serial of unlabeled training datasets {Dt

train}T

t=1

are sequentially provided, where

train ={xt

i}Nt

i=1

indicates the dataset at time

. Though unlabeled,

we denote the known or potential appeared categories until time

Ct={1,2, ..., Kt}

. The model

is expected to discover newly appeared unknown categories from

train

, store the representations of

the discovered novel categories and maintain the knowledge of the known categories.

2.2 Evaluation Metrics

At each time-step

, we evaluate the classiﬁcation performance on the test dataset

test =

{(xs

i, ys

i)|s≤t}

, containing test samples from all the known or previously discovered categories. For

the newly appeared unknown categories, we evaluate the novel category discovery performance. The

maximum forgetting metric Mf

and the

ﬁnal discovery metric Md

are designed for evaluation.

To evaluate the performance of the clustering assignments, we follow the standard practise [

] to

adopt clustering accuracy on the known categories and the novel categories, denoted as

ACCt

known

and

ACCt

novel

, respectively. The maximum forgetting

is deﬁned as the maximum value of the

differences between

ACC0

known

and

ACCt

known

for every

and the ﬁnal discovery

is deﬁned

as the ﬁnal cluster accuracy on novel categories, i.e.,

Md=ACCT

novel

. See more details in the

supplementary materials.

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

GrowandMerge:AUniedFrameworkforContinuousCategoriesDiscoveryXinweiZhang1,JianwenJiang2,YutongFeng2,Zhi-FanWu2,XibinZhao1y,HaiWan1,MingqianTang2,RongJin2,YueGao1,3y1BNRist,KLISS,SchoolofSoftware,TsinghuaUniversity2AlibabaGroup3THUIBCS,BLBCI,TsinghuaUniversityxinwei-z21@mails.tsinghua.edu.cn{jianwe...

展开>> 收起<<

Grow and Merge A Uniﬁed Framework for Continuous Categories Discovery Xinwei Zhang1 Jianwen Jiang2 Yutong Feng2 Zhi-Fan Wu2 Xibin Zhao1y.pdf

共13页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Grow and Merge A Uniﬁed Framework for Continuous Categories Discovery Xinwei Zhang1 Jianwen Jiang2 Yutong Feng2 Zhi-Fan Wu2 Xibin Zhao1y

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: