Cloth Funnels Canonicalized-Alignment for Multi-Purpose Garment Manipulation Alper Canberk1 Cheng Chi1 Huy Ha1

2025-04-29 0 0 5.66MB 8 页 10玖币

侵权投诉

Cloth Funnels: Canonicalized-Alignment

for Multi-Purpose Garment Manipulation

Alper Canberk 1, Cheng Chi 1, Huy Ha 1,

Benjamin Burchﬁel 2, Eric Cousineau 2, Siyuan Feng 2and Shuran Song 1

clothfunnels.cs.columbia.edu

Abstract— Automating garment manipulation is challenging

due to extremely high variability in object conﬁgurations.

To reduce this intrinsic variation, we introduce the task

of “canonicalized-alignment” that simpliﬁes downstream

applications by reducing the possible garment conﬁgurations.

This task can be considered as “cloth state funnel” that

manipulates arbitrarily conﬁgured clothing items into a

predeﬁned deformable conﬁguration (i.e. canonicalization) at

an appropriate rigid pose (i.e. alignment). In the end, the cloth

items will result in a compact set of structured and highly

visible conﬁgurations – which are desirable for downstream

manipulation skills. To enable this task, we propose a novel

canonicalized-alignment objective that effectively guides

learning to avoid adverse local minima during learning. Using

this objective, we learn a multi-arm, multi-primitive policy that

strategically chooses between dynamic ﬂings and quasi-static

pick and place actions to achieve efﬁcient canonicalized-

alignment. We evaluate this approach on a real-world ironing

and folding system that relies on this learned policy as the

common ﬁrst step. Empirically, we demonstrate that our

task-agnostic canonicalized-alignment can enable even simple

manually-designed policies to work well where they were pre-

viously inadequate, thus bridging the gap between automated

non-deformable manufacturing and deformable manipulation.

I. INTRODUCTION

Why has garment manipulation proved more difﬁcult to

automate than more typical rigid and articulated objects? We

argue that two key factors are severe self occlusion, which is

present in the large set of possible crumpled states, and the

inﬁnite degrees of freedom inherent to clothing. As a result,

it is impractical to manually deﬁne manipulation policies that

achieve reliable manipulation — a cornerstone of current auto-

mated non-deformable manufacturing pipelines. In this work,

we explore bridging the gap between existing approaches to

automation and the challenging domain of clothing. We show

that when a robot ﬁrst manipulates arbitrarily conﬁgured

clothing items into a predeﬁned conﬁguration (i.e. canonical-

ization) at an appropriate pose (i.e. alignment), downstream

manipulation skills work signiﬁcantly more reliably.

Recently, real-world cloth manipulation has received

signiﬁcant attention. Some of the earliest cloth manipulation

work explored manually designed heuristics which worked

well for speciﬁc clothing types, conﬁgurations, and tasks,

such as cloth unfolding [

–

], smoothing [

], folding [

–

], but their strong assumptions initial states, ﬁducial markers,

specialized tools, or cloth type/shape do not generalize. More

recently, learning-based approaches have shown success in

more general cloth manipulation behavior. One line of work

1Columbia University

2Toyota Research Institute

Fig. 1. Canonicalized-Alignment funnels the large space of possible cloth

conﬁgurations into a much smaller and better structured set of highly-visible

states that greatly simpliﬁes downstream tasks such as ironing or folding.

has explored supervised-learning from human demonstrations

for smoothing [

] and folding [

], but those methods required

costly human demonstrations/annotations. Another recent line

of work employs fully self-supervised learning and has shown

success in learning to unfold [

] (but doesn’t generalize

to other tasks) and in tackling visual goal-conditioned

manipulation of a single square cloth instance [12].

Instead of learning arbitrary monolithic cloth manipulation

tasks, we hypothesize that it is more efﬁcient to learn a

robust task-agnostic canonicalization and alignment policy

from which other task-speciﬁc manipulation skills may be

chained. This is because such a policy funnels unstructured

and self-occluded cloth conﬁgurations into structured states

with clearly visible key points (Fig. 1, middle), reducing the

complexity of the task-speciﬁc downstream policy, and en-

abling even simple heuristics to work with a high success rate.

To this end, we deﬁne a new “canonicalized-alignment” task

for garment manipulation, where the goal is to transform a gar-

ment from its arbitrary initial state into a canonical shape (de-

ﬁned by its category) and align it with a particular 2D transla-

tion and rotation. The end result is a decomposition of garment

manipulation into two factorized parts. The ﬁrst part funnels

a diverse set of cloth conﬁgurations into SE-2 transforms of a

small set of states with high visibility. The second part consists

of downstream behaviors which relies on kinematically

feasible transforms of structured initial conﬁgurations and

full keypoint observability to achieve high task success rates.

Our primary contribution is the introduction of

arXiv:2210.09347v1 [cs.RO] 17 Oct 2022

Fig. 2.

Approach Overview.

a) A batch of scaled and rotated observations are created from a top-down RGB image of the workspace and then concatenated

with a scale-invariant coordinate map. b) The batch of inputs is fed through the factorized network architecture, producing a batch of rotated and scaled

value maps for each primitive. c) All primitive batches are concatenated and the maximum value pixel parameterizes the action to be executed.

“canonicalized-alignment”, a garment manipulation task

which serves as a cloth funnel for reducing general-purpose

garment manipulation complexity. We achieve this by the

following technical contributions:

•

We propose a learned multi-arm, multi-primitive

manipulation policy that strategically chooses between

dynamic ﬂings and quasi-static pick&place actions to

efﬁciently and precisely transform the garment into its

canonicalized and aligned conﬁguration.

•

To train the policy, we proposed a novel factorized reward

function that avoids adverse local minima which plague

the generic goal-reaching formulations by decoupling

deformable shape and rigid pose.

•

We evaluate our approach in multiple downstream garment

manipulation tasks in the real-world on a physical robot,

including folding and ironing.

Our experiments show that incorporation of canonicalized-

alignment signiﬁcantly reduces the complexity of downstream

applications, suggesting that robust canonicalized-alignment

provides a practical step forward toward multi-purpose

garment manipulation from arbitrary states for diverse tasks.

II. RELATED WORK

Heuristic-based cloth manipulation.

Heuristic-based

manipulation pipelines – where action selection and planning

is manually designed – can produce impressive results.

However, the generality and robustness of these approaches is

limited due to strong assumptions regarding pre-canonicalized

initial state [

], ﬁducial markers [

], specialized tools [

and cloth type and shape [1,2,4–6,15–18].

Learning-based cloth unfolding.

Learning-based methods

can self-discover the best policies for a distribution of

cloths using real-world self-supervision [

] or simulator

states [

]. While these approaches have been successfully

applied to cloth unfolding [

] or canonicalization [

they do not consider canonicalized-alignment. This limits

their applicability since heuristic-based pipelines cope poorly

with unmet cloth assumptions or kinematic constraints.

Goal-conditioned cloth manipulation.

Towards generic

goal-conditioned cloth manipulation, prior works have

investigated reinforcement learning [

–

], real-world self-

supervised learning [

] and imitation learning [

]. However,

these methods often struggle to bridge the sim2real gap [

generalize across cloth instances [

] or generalize

between garment types [

]. Furthermore, all

goal-conditioned works do not address how goal vertices/key

points/images can be obtained for a completely novel cloth

instance. Instead, our proposed approach can accommodate

different garment categories and generalize to a variety of

novel real-world garment instances from simulation training.

III. METHOD

A. A Multi-Purpose Garment Manipulation Pipeline

We propose a factorized approach to multi-purpose

garment manipulation from arbitrary states that decomposes

the process into two steps. First, the robot executes a learned

task-agnostic canonicalized-alignment policy, which leaves

the garment in a known conﬁguration predeﬁned for the

clothing category at a speciﬁed 2D rotation and translation.

Second, the robot executes a task-speciﬁc keypoint based

policy, which could be as simple as a manually-designed

heuristic. This approach confers three primary beneﬁts:

•Arbitrary initial conﬁguration

: Canonicalization funnels

the large space of possible cloth conﬁgurations into a

narrow distribution of highly-structured fully-observable

conﬁgurations from which downstream policies can more

easily operate.

•Downstream task-awareness

: Flexible goal-conditioned

alignment allows the canonicalized cloths to be placed at

speciﬁed positions and orientations that are kinematically

appropriate for particular downstream tasks.

•Clothing category generalization

: A keypoint-based cloth

representation effectively reduces the observation space

from having to represent the inﬁnite DoF down to a few

meaningful keypoints. Further, cloths are always in a known

canonicalized conﬁguration. These two properties combined

not only simpliﬁes learning downstream task-speciﬁc ma-

nipulation policies, but also makes it possible to engineer

heuristics that work reliably for a clothing category.

Next, we will discuss how the canonicalized-alignment task is

formulated (Sec. III-B), learned (Sec. III-C), and implemented

alongside the several downstream task policies (Sec. III-D).

B. The Canonicalization & Alignment Task

Problem Formulation.

Given a clothing item in some

clothing category in an arbitrary initial conﬁguration, the goal

of canonicalization is to reach the human-deﬁned standard de-

formable conﬁguration for that clothing category, such as a T-

shaped conﬁguration for shirts and the upside-down V-shaped

conﬁguration for pants. Note that this only accounts for the

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

ClothFunnels:Canonicalized-AlignmentforMulti-PurposeGarmentManipulationAlperCanberk1,ChengChi1,HuyHa1,BenjaminBurchel2,EricCousineau2,SiyuanFeng2andShuranSong1clothfunnels.cs.columbia.eduAbstractAutomatinggarmentmanipulationischallengingduetoextremelyhighvariabilityinobjectcongurations.Toreduceth...

展开>> 收起<<

Cloth Funnels Canonicalized-Alignment for Multi-Purpose Garment Manipulation Alper Canberk1 Cheng Chi1 Huy Ha1.pdf

共8页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Cloth Funnels Canonicalized-Alignment for Multi-Purpose Garment Manipulation Alper Canberk1 Cheng Chi1 Huy Ha1

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: