Planning Assembly Sequence with Graph Transformer Lin Ma1 Jiangtao Gong1B Hao Xu2 Hao Chen2 Hao Zhao1 Wenbing Huang1and Guyue Zhou1 Abstract Assembly sequence planning ASP is the essential

2025-05-02 0 0 3.18MB 7 页 10玖币

侵权投诉

Planning Assembly Sequence with Graph Transformer

Lin Ma1, Jiangtao Gong1B, Hao Xu2, Hao Chen2, Hao Zhao1, Wenbing Huang1and Guyue Zhou1

Abstract— Assembly sequence planning (ASP) is the essential

process for modern manufacturing, proven to be NP-complete

thus its effective and efﬁcient solution has been a challenge

for researchers in the ﬁeld. In this paper, we present a graph-

transformer based framework for the ASP problem which is

trained and demonstrated on a self-collected ASP database. The

ASP database contains a self-collected set of LEGO models.

The LEGO model is abstracted to a heterogeneous graph

structure after a thorough analysis of the original structure

and feature extraction. The ground truth assembly sequence

is ﬁrst generated by brute-force search and then adjusted

manually to in line with human rational habits. Based on this

self-collected ASP dataset, we propose a heterogeneous graph-

transformer framework to learn the latent rules for assembly

planning. We evaluated the proposed framework in a series

of experiment. The results show that the similarity of the

predicted and ground truth sequences can reach 0.44, a medium

correlation measured by Kendall’s τ. Meanwhile, we compared

the different effects of node features and edge features and

generated a feasible and reasonable assembly sequence as a

benchmark for further research. Our data set and code is

available on https://github.com/AIR-DISCOVER/ICRA ASP.

I. INTRODUCTION

To facilitate automatic assembly in a wide range of ﬁelds,

such as the furniture industry, auto manufacturing industry,

and arts and crafts industry, assembly sequence planning is

a key procedure after product design.

The dominant methods for ASP problems are assembly-

by-disassembly planning and the combinatorial optimization

algorithm. To be detailed, the main task of Assembly-by-

disassembly planning is ﬁnding a feasible removing path by

recursively removing parts from the assembly product [1] [2].

Moreover, the combinatorial optimization algorithm aims at

determining the precedence relationship among all parts un-

der complex constraints [3] [4] using dynamic programming.

ASP is proven to be NP-complete, so its effective and

efﬁcient solution has been a challenge for researchers in the

ﬁeld. Consequently, there is much room for progress in this

research. Deep learning methods have been widely used in

various areas and proven to be effective, so we expect to

use deep learning methods to learn the latent rules of an

assembly problem from previous experience. However, the

two dominant methods mentioned above can only ﬁnd a

solution for a speciﬁc assembly model. As a result, there

is no dataset specialized for ASP problems to train a deep

learning model.

The datasets most related to our task are illustrated as fol-

lows. A main application of ASP is generating step-by-step

1Institute for AI Industry Research (AIR), Tsinghua

University, 10080, Haidian District, Beijing, P.R.China.

lastnamefirstname@air.tsinghua.edu.cn

2Qianzhi Technology, China hao.xu.chn@gmail.com

Fig. 1. An example of Assembly sequence planning

instructions for furniture and household articles, which are

also universally used in many research [5]. 80 IKEA furniture

models are present and applied to assembly environment

design in [6]. These furniture models are made up of several

parts without complex block relationships, so the main task

is to ensure stability or visibility in the assembly process.

Moreover, a large-scale part-level 3D dataset containing var-

ious objects in indoor scenes was released [7]. This dataset

enables many 3D generative tasks at the part level, such as

estimating the location and pose of parts [8] and learning a

consistent part order for a given object category [9]. Except

for furniture and household items, 3D shapes simulated

by the LEGO model are generated via sequence assembly

[10] [11]. However, the aforementioned assembly datasets

are mostly designed or collected for generative tasks rather

than sequence assembly planning, so a dataset speciﬁcally

for ASP problem is demanded. A suitable assembly dataset

composed of a moderate number of instances and containing

common block relationships would provide us with more

opportunities to establish universal solutions to the ASP

problem.

However, the collection of assembly instances is no easy

task for the following reasons. Firstly, the instances are

supposed to be sufﬁcient for training and homogeneous with

each other. Nevertheless, the majority of the industries only

release a few temples. Secondly, even if we have the temples,

the assembly sequences, namely, the ground truth, are always

not available.

In this paper, we present a collective LEGO dataset for the

ASP task. Structures built from LEGO are complex enough

to approach real-world design and have a large number of

models designed and uploaded by users. Unfortunately, not

all models are practical, so we made some adjustments,

which will be illustrated in section III. Moreover, we propose

a graph-transformer-based framework for ASP. Our goal is

arXiv:2210.05236v3 [cs.AI] 15 Oct 2022

to train a network to learn the latent rules of the assembly

sequence and automatically generate a feasible assembly

sequence according to a design template.

The main contribution of this work can be summarized as

follows.

1) A self-collected ASP dataset using user-deﬁned models

from LEGO Studio and feasible assembly sequences

by brute force method and manual adjustment as

ground truth;

2) A graph-transformer based framework for ASP prob-

lem, with a heterogeneous graph attention network

to encoder the models, which are decoded with the

attention mechanism to generate assembly sequences;

3) A series of experiment to evaluate the effectiveness of

the proposed framework and offer the benchmark for

this ASP problem.

II. RALATED WORK

Assembly sequence planning ASP problems are usu-

ally solved by assembly-by-disassembly planning and the

combinatorial optimization algorithm. The assembly-by-

disassembly method ﬁnds a feasible disassembly order via

motion planning [12]. Being NP-complete, it requires numer-

ous search operations even to ﬁnd a feasible disassembly

path. Thus researchers compress the search space using a

tree structure, such as ”Rapidly-exploring random tree” [13]

and Expansive Vornoi tree [14]. Nevertheless, the combinato-

rial optimization algorithm always implements the heuristic

methods, such as Ant Colony optimization algorithm [15],

Gray Wolf optimization algorithm [16], and Artiﬁcial Neural

Networks [17].

ASP problems have not been extensively studied by deep

learning methods. To the best of our knowledge, the only

method related to deep learning is proposed in [18]. We

believe there is still much room for progress in this research.

LEGO model Since LEGO models are complex enough

to approach real-world design, it is common for researchers

and designers to regard the LEGO model as an abstraction

of a physical object. By studying the LEGO model, we can

increase our cognition of the real world and be inspired

to solve practical problems. For example, LEGO models

are used to approximate actual 3D shapes while ensuring

that the ﬁnal products have some good properties such as

connectivity and stability [19] [20]. Furthermore, the LEGO

models abstracted from physical objects are applied to the

deep generative models for 3D shape generative tasks [10]

[11].

Besides, as an abstraction of real-world objects, the LEGO

model is a powerful tool for promoting and testing human

cognitive abilities[21]. Moreover, cognitive loads are usually

studied during LEGO assembly tasks [22].

In the above scenario, the LEGO model is abstracted

into LEGO Model Representation graph [23] [10], with a

single vertex in the graph representing a LEGO brick (regular

brick, plate, or tile) and an edge representing the linkage

between two LEGO bricks. This is consistent with real-

Fig. 2. Overview of the LEGO models.

world assembly problems. For example, furniture and cars

are abstracted to graph structures [7] [1].

Graph-transformer The transformer has became the stan-

dard architecture for a wild variety of ﬁelds. In particular,

its variants have been modiﬁed to explore the latent feature

of graph-structured data and fulﬁll different tasks for natural

language processing [24], biological molecular structure [25]

[26], social networks [27], and academic network [28].

In a heterogeneous graph, different types of objects are

mapped to different types of nodes, and the same is true

for edges [29] [30]. This abstraction is more appropriate for

containing more complete information. Accordingly, LEGO

models are similarly devised to heterogeneous graphs in

our data set. Different node types represent different LEGO

bricks and different edge types for different relations between

LEGO bricks.

Almost all of the tasks of the graph-transformer framework

are node classiﬁcation, graph classiﬁcation, link prediction,

and text generation, but barely see its application in sequence

planning. To the best of our knowledge, this is the ﬁrst

attempt to utilize graph transformers in an assembly sequence

planning task.

III. LEGO-ASP DATASET

A. Data set summary

We collect 100 LEGO animal models created and up-

loaded by individual users in LEGO Studio, among which

the simplest one is composed of 3 bricks and the most

complex one is composed of 44 bricks. The median number

of brick numbers in a LEGO model is 19. As shown in Fig.

2, we present four models with different number of bricks

as examples. And the statistics of the number of bricks in a

individual LEGO model is demonstrated in the histogram.

The total number of bricks in all LEGO models is 2127,

of which 23 types of bricks are frequently used. The rest are

used only a few times or customized, but they take up a non-

negligible percentage as a whole. According to the role of

bricks, the frequently-used bricks can be roughly categorized

into the following three types: regular brick, plate, and tile.

In addition, they can also be categorized into four types

according to their shape: cubic, curve, slope, and round. At

the same time, the bricks are of different sizes. With different

combinations of these features, a large variety of bricks exist.

We present different types of LEGO bricks in Fig. 3. For

example, the ﬁrst brick is a regular brick as well as round

type. The second is a customized brick, the seventh is a plate

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

PlanningAssemblySequencewithGraphTransformerLinMa1,JiangtaoGong1B,HaoXu2,HaoChen2,HaoZhao1,WenbingHuang1andGuyueZhou1AbstractAssemblysequenceplanning(ASP)istheessentialprocessformodernmanufacturing,proventobeNP-completethusitseffectiveandefcientsolutionhasbeenachallengeforresearchersintheeld.Inth...

展开>> 收起<<

Planning Assembly Sequence with Graph Transformer Lin Ma1 Jiangtao Gong1B Hao Xu2 Hao Chen2 Hao Zhao1 Wenbing Huang1and Guyue Zhou1 Abstract Assembly sequence planning ASP is the essential.pdf

共7页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Planning Assembly Sequence with Graph Transformer Lin Ma1 Jiangtao Gong1B Hao Xu2 Hao Chen2 Hao Zhao1 Wenbing Huang1and Guyue Zhou1 Abstract Assembly sequence planning ASP is the essential

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: