Discovering Design Concepts for CAD Sketches Yuezhi Yang The University of Hong Kong

2025-05-03 0 0 2.21MB 24 页 10玖币

侵权投诉

Discovering Design Concepts for CAD Sketches

Yuezhi Yang∗

The University of Hong Kong

Microsoft Research Asia

yzyang@cs.hku.hk

Hao Pan

Microsoft Research Asia

haopan@microsoft.com

Abstract

Sketch design concepts are recurring patterns found in parametric CAD sketches.

Though rarely explicitly formalized by the CAD designers, these concepts are

implicitly used in design for modularity and regularity. In this paper, we propose a

learning based approach that discovers the modular concepts by induction over raw

sketches. We propose the dual implicit-explicit representation of concept structures

that allows implicit detection and explicit generation, and the separation of structure

generation and parameter instantiation for parameterized concept generation, to

learn modular concepts by end-to-end training. We demonstrate the design concept

learning on a large scale CAD sketch dataset and show its applications for design

intent interpretation and auto-completion.

1 Introduction

Parametric CAD modeling is a standard paradigm for mechanical CAD design nowadays. In

parametric modeling, CAD sketches are fundamental 2D shapes used for various 3D construction

operations. As shown in Fig. 1, a CAD sketch is made of primitive geometric elements (e.g. lines,

arcs, points) which are constrained by different relationships (e.g. coincident, parallel, tangent); the

sketch graph of primitive elements and constraints captures design intents, and allows adaptation and

reuse of designed parts by changing parameters and updating all related elements automatically [

Designers are therefore tasked with the meticulous design of such sketch graphs, so that the inherent

high-level design intents are easy to interpret and disentangle. To this end, meta-structures (Fig. 1),

which we call sketch concepts in this paper, capture repetitive design patterns and regulate the design

process with more efﬁcient intent construction and communication [

]. Concretely, each sketch

concept is a structure that encapsulates speciﬁc primitive elements and their compositional constraints,

and the interactions of its internal elements with outside only go through the interface of the concept.

How to discover these modular concepts automatically from raw sketch graphs? In this paper, we cast

this task as a program library induction problem by formulating a domain speciﬁc language (DSL) for

sketch generation, where a sketch graph is formalized as a program, and sketch concepts are modular

functions that abstract primitive elements and compose the program (Fig. 1). Discovering sketch

concepts thus becomes the induction of library functions from sketch programs. While previous

works address the general library induction problem via expensive combinatorial search [

we present a simple end-to-end deep learning solution for sketch concepts. Speciﬁcally, we bridge

the implicit and explicit representations of sketch concepts, and separate concept structure generation

from parameter instantiation, so that a powerful deep network can detect and generate sketch concepts,

by training with the inductive objective of reconstructing sketch with modular concepts.

We conduct experiments on large-scale sketch datasets [

]. The learned sketch concepts show that

they provide modular interpretation of design sketches. The network can also be trained on incomplete

input sketches and learn to auto-complete them. Comparisons with state-of-the-art approaches that

∗Work done during internship at Microsoft Research Asia.

36th Conference on Neural Information Processing Systems (NeurIPS 2022).

arXiv:2210.14451v1 [cs.LG] 26 Oct 2022

Coinc. Coinc.

Coinc.

In arg: 1

Arc

Out arg: 0

Tangent

Out arg: 1

Coinc.

Arc

Out arg:1

Tangent

Coinc.

Out arg:0 Coinc.

Line

Parallel

Distance

Perpend.

Horizon.

Tangent

Line

Coinc.

Line

Coinc.

Arc

Parallel

Distance

Arc

Coinc.

Line

Perpend

Coinc. Coinc.

Horizon.

Tangent

Coinc.

In arg:0

(a) S.t0:T1

S.t1:T1

S.t2:T1

(b) Learned program that restructures the sketch:

0→λ(αo

0, αo

1).{t0:Arc, t1:Tang., t2, t3:Coinc., R={t1(t0, αo

0), t2(t0, αo

0), t3(t0, αo

1)}}

1→λ(αi

0, αi

1).{t0,t1,t2,t3:Line, t4,t5,t6,t7:Coinc., t8:Perpend., t9:Parallel, t10 :Distance, t11 :Horizon,

R={t4(t0, t3), t5(t0, t2), t0(t1, t2), t7(t1, t3), t8(t1, t2), t9(t2, t3), t10 (t2, t3), t11 (t3), αi

0(t1), αi

1(t0)}}

S→{t0,t1:T1

0,t2:T1

1, R={t0(t2.αi

1, t2.αi

0), t1(t2.αi

0, t2.αi

1)}}

Figure 1:

Concept learning from sketch graphs

(a)

In black are the raw sketch and its constraint

graph, with nodes showing primitives and edges depicting constraints. Colored are the restructured

sketch and its modular constraint graph, where each module box represents a concept; primitives and

constraint edges are colored according to the modular concepts.

(b)

The restructured sketch graph in

our DSL program representation (List 1), where the whole sketch

is compactly constructed with

three instances of two learned L1types. We simplify notation super/sub-scripts for readability.

solve sketch graph generation through autoregressive models show that the modular sketch concepts

learned by our approach enable more accurate and interpretable completion results.

To summarize, we make the following contributions in this paper:

•

We formulate the task of discovering modular CAD sketch concepts as program library

induction for a declarative DSL modeling sketch graphs.

•

We propose a self-supervised deep learning framework that discovers modular libraries for

the DSL with simple end-to-end training.

•

We show the framework learns from large-scale datasets sketch concepts capturing intuitive

and reusable components, and enables structured sketch interpretation and auto-completion.

2 Related work

Concept discovery for CAD sketch

It is well acknowledged in the CAD design community that

design intents are inherent to and implicitly encoded by the combinations of geometric primitives and

constraints [

]. However, there is generally no easy approach to discover the intents and make

them explicit, albeit through manual design of meta-templates guided by expert knowledge [

We propose an automatic approach to discover such intents, by formulating the intents as modular

structures with self-contained references, and learning them through self-supervised inductive training

with simple objectives on large raw sketch dataset. Therefore, we provide an automatic approach for

discovering combinatorially complex structures through end-to-end neural network learning.

Generative models for CAD sketch

A series of recent works [

] use autoregressive

models [

] to generate CAD sketches and constraints modeled through pointer networks [

]. These

works focus on learning from large datasets [

] to generate plausible layouts of geometric primitives

and their constraints, which can then be ﬁne-tuned with a constraint solver for more regular sketches.

Different from these works, our aim is to discover modular structures (i.e. sketch concepts) from the

concrete sketches. Therefore, our framework provides higher-level interpretation of raw sketches and

more transparent auto-completion than these works (cf. Sec. 6).

Program library induction for CAD modeling

Program library induction has been studied in

the shape modeling domain [

]. General program synthesis assisted by deep learning is a research

topic with increasing popularity [

]. The library induction task speciﬁcally involves

combinatorial search, as has been handled by neural guided search [

] or by pure stochastic

sampling [

]. We instead present an end-to-end learning algorithm for sketch concept induction.

In particular, based on key observations about sketch concepts, we present implicit-explicit dual

representations of concept library functions, and separate the concept structure generation from

parameter instantiation, to enable self-supervised training with induction objectives.

List 1: A domain-speciﬁc language formulating CAD sketch concepts

// Basic data types

Length, Angle, Coord, Ref

// L0primitive types

Line →cstart_x, cstart_y, cend_x, cend_y: Coord

Circle →ccenter_x, ccenter_y: Coord, lradius : Length

· · ·

// L0constraint types

Coincident →λ(r1, r2:Ref).{}

Parallel Distance →λ(r1, r2:Ref).{ldist : Length}

· · ·

// L1composite types

i→λ([αk:Ref]).{t0

i,j :T0

j∈L0, RT1

i[t0

i,j ]∪[αk]}

// Sketch decomposition

S→ {t1

i:T1

i∈L1, RS([t1

i])}

3 CAD sketch concept formulation

To capture the notion of sketch concepts precisely, we formulate a domain speciﬁc language (DSL)

(syntax given in List 1, an exhaustive list of data types given in the supplementary). In the DSL, we

ﬁrst deﬁne the basic data types, including length, angle, coordinate, and the reference type, where a

reference binds to another reference or a primitive for modeling the constraint relationships. Second,

we deﬁne the

collection of primitive and constraint types as given in raw sketches. In particular,

we regard the constraints as functions whose arguments are the references to bind with primitives, e.g.

a coincident constraint

c=λ(r1, r2:Ref).{}

, where a function is represented in the lambda calculus

style (one may refer to [

] for introductory lambda calculus formality). Some constraints have

parameters other than mere references, which are treated as variables inside, e.g. parallel distance in

List 1

. Third, we deﬁne the sketch concepts as

types composed of

types. To be speciﬁc, a

composite type

i∈L1

is a function with arguments

[αk]

and members

i,j :T0

j∈L0

, which are

connected through a composition operator

RT1

i={p(q)|p, q ∈[t0

i,j ]∪[αk]}

that speciﬁes how each

pair of primitive elements binds together. For example, a coincident constraint

p=λ(r1, r2).{}

may

take a line primitive

as its ﬁrst argument and bind to an argument

αk

of the composite type as its

second argument, i.e.

p(q, αk)∈RT1

; on the other hand, an argument

αk

may bind to a primitive

, which is speciﬁed by

αk(q)∈RT1

. Finally, an input sketch

is restructured as a collection of

composite types

i:T1

i∈L1

, as well as their connections speciﬁed by a corresponding composition

operator

records how different concepts bind through their arguments, which further transfers

typed elements inside the concepts and translates into the raw constraint relationships of the

sketch graph. Fig. 1(b) shows an example DSL program encoding sketches and concepts.

Given the explicit formulation of CAD sketches through a DSL, the discovery of sketch concepts

becomes the task of learning program libraries

by induction on many sketch samples. Therefore,

our task resembles shape program synthesis that aims at building modular programs for generating

shapes [

], and differs from works that use autoregressive language models to generate CAD sketch

programs one token at a time [

]. In Sec. 6.2, we show that the structured learning of CAD

sketches enables more robust auto-completion than unstructured language modeling.

The search of structured concepts is clearly a combinatorial problem with exponential complexity,

which is intractable unless we can exploit the inherent patterns in large-scale sketch datasets. However,

to enable deep learning based detection and search of structured concepts, we need to bridge the

implicit deep representations and the explicit and interpretable structures, which we build through the

following two key observations:

•A concept has dual representations

: implicit and explicit. The implicit representation as

embeddings in latent spaces is compatible with deep learning, while the explicit representa-

tion provides structures on which desired properties (e.g. modularity) can be imposed.

While other works [

] have skipped such constraints, we preserve them but omit generating the

parameter values that can be reliably deduced from primitives. See more discussions in the supplementary.

input sketch S=[t0

[e0

]

[qi]

concept lib L1structure T1

instance t1

[t1

assemble

generation

loss

target

Struct

Param

Encoder Decoder

(a) detection module (b) generation module (c) loss computation

Figure 2:

Framework illustration

(a)

The detection module is a transformer network that detects

from the sketch sequence

[t0

implicitly encoded concepts

[qi]

and their composition

(b)

Each

is quantized against the concept library

to obtain prototype

, which is expanded by structure

network into an explicit structure

and further instantiated by parameter network into

(c)

The

collection of

[t1

are assembled by composition operator

generated from

to obtain the ﬁnal

generated sketch graph, which is compared with the input sketch for loss computation.

•A concept is a parameterized structure

. A concept is a composite type with ﬁxed modular

structure for interpretability, but the structure is always instantiated by assigning parameters

to its component primitives when the concept is found in a sketch.

3.1 Method overview

According to the two observations, we design an end-to-end sketch concept learning framework by

self-supervised induction on sketch graphs. As shown in Fig. 2, the framework has two main steps

before loss computation: a detection step that generates implicit representations of concepts making

up the input sketch, and an explicit generation step that expands the implicit concepts into concrete

structures on which self-supervision targets like reconstruction and modularity are applied.

Building on a state-of-the-art detection architecture [

], the detection module

takes a sketch

input and detects the modular concepts within it, i.e.

{qi}=D(S, {qi})

, where the concepts are

represented implicitly as latent codes

{qi}

, and

{qi}

are a learnable set of concept instance queries.

Notably, we apply vector quantization to the latent codes and obtain

{q0

i= minp∈L1||p−qi||2}

which ensures that each concept is selected from the common collection of learnable concepts

used for restructuring all sketches.

The explicit generation module is separated into two sub-steps, structure generation and parameter

instantiation, which ensures that the modular concept structures are explicit and reused throughout

different sketch instances. Speciﬁcally, the structure network takes each quantized concept code

and generates its explicit form

in terms of primitives and constraints of

types along with the

composition operator

RT1

. Subsequently, the parameter network instantiates the concept structure by

assigning parameter values to each component of

conditioned on

and input sketch, to obtain

The composition operator

for combining

{t1

is generated from a special latent code

trans-

formed by Dfrom a learnable token qRappended to {qi}.

The entire model is trained end-to-end by reconstruction and modularity objectives. In particular,

we design loss functions that measure differences between the generated and groundtruth sketch

graphs, in terms of both per-element attributes and pairwise references. Given our explicit modeling

of encapsulated structures of the learned concepts, we can further enhance the modularity of the

generation by introducing a bias loss that encourages in-concept references.

4 End-to-end sketch concept induction

4.1 Implicit concept detection

Sketch encoding

A raw sketch

can be serialized into a sequence of

primitives and constraints.

Previous works have adopted slightly different schemes to encode the sequence [

In this paper, we build on the previous works and take a simple strategy akin to [

] for input

sketch encoding. Speciﬁcally, we split each

typed instance

into several tokens: type,parameter,

and a list of references. For each of the token category, we use a speciﬁc embedding module. For

example, parameters as scalars are quantized into ﬁnite bins before being embedded as vectors (see

supplementary for the quantization details), and since there are at most ﬁve parameters for each

primitive, we pack all parameter embeddings into a single code. On the other hand, each constraint

reference as a primitive index is directly embedded as a code. Therefore, each token of a

typed

instance is encoded as

et0.x =enctype(t0) + encpos(t0.x) + encparam(t0.x)|encref (t0.x),(1)

where

t0.x

iterates over the split tokens (i.e., type, parameters, references), the type embedding is

shared for all tokens of the instance, the position embedding counts the token index in the whole

split-tokenized sequence of

, and parameter or reference embeddings are applied where applicable.

Concept detection

We build the detection network as an encoder-decoder transformer following

[

]. The transformer encoder operates on the sketch encoded sequence

[et0

i∈S]

and produces the

contextualized sequence

[e0

i∈S]

through layers of self-attention and feed-forward. The transformer

decoder takes a learnable set of concept queries

[qi]

of size

kqry

plus a special query

for

composition generation, and applies interleaved self-attention, cross-attention to

[e0

and feed-

forward layers to obtain the implicit concept codes

[qi]

and

. The concept codes are further

quantized into

[q0

by selecting concept prototypes from a library

implicitly encoding

, before

being expanded into explicit forms.

4.2 Explicit concept structure generation

Concept structure expansion

Given a library code

q0∈L1

representing a type

T1∈L1

, through

an MLP we expand its explicit structure as a collection of codes

[t0

representing the

type instances

[t0

i]and a matrix representing the composition RT1of [t0

i]and arguments (cf. List 1).

concept A concept B

primitive constraint

inward arg outward arg

We ﬁx the maximum number of

type instances to

kL0

(12 by default), and

split the arguments into two groups, inward arguments and outward arguments,

each of maximum number

karg

(2 by default). Each type code

is decoded

into discrete probabilities over

with an additional probability for null type

to indicate the emptiness of this element (cf. Sec. 5.1), by

dectype(·)

the inverse of

enctype(·)

in Sec. 4.1. An inward argument only points to a

primitive inside the concept structure and originates from a constraint outside,

and conversely an outward argument only points to primitives outside and

originates from a constraint inside the concept (see inset for illustration); the split into two groups

eases composition computation, as discussed below.

The composition operator

RT1

is implemented as an assignment matrix

RT1

of shape

(2kL0+karg )×

(kL0+karg )

, where each row corresponds to a constraint reference or inward argument, and each

column to a primitive or outward argument. The two-fold coefﬁcient of constraint references comes

from that any constraint we considered in the dataset [

] has at most two arguments. Each row is a

discrete probability distribution such that

PjRT1[i, j] = 1

, with the maximum entry signifying that

the

-th constraint/outward argument refers to the

-th primitive/inward argument. We compute

RT1

by ﬁrst mapping the concept code

to a matrix of logits in the shape of

RT1

, and then applying

softmax transform for each row. Notably, we avoid the meaningless loops of an element referring

back to itself, and inward arguments referring to outward arguments, by masking the diagonal blocks

RT1[2i:2i+2, i], i∈[kL0]and the argument block RT1[2kL0:, kL0:] by setting their logits to −∞.

Cross-concept composition

Aside from references inside a concept, references across concepts are

generated to complete the whole sketch graph. We achieve cross-concept references by argument

passing (see inset above for illustration). In particular, we implement the cross-concept composition

operator

as an assignment matrix

of shape

(kqry·karg)×(kqry ·karg)

directly mapped from

through an MLP. Similar to the in-concept composition matrix, each row of the cross-concept matrix

is a discrete distribution such that

PjRS[i, j]=1

, with the maximum entry signifying that the

(imod karg )

-th outward argument of the

bi/karg c

-th concept instance refers to the

(jmod karg )

-th

inward argument of the bj/kargc-th concept instance.

The complete cross-concept reference is therefore the product of three transport matrices:

Rcref [t1

i, t1

j] = Rt1

i[:2kL0, kL0:]×RS[i·karg :(i+1)·karg, j·karg:(j+1)·karg]×Rt1

j[2kL0:,:kL0],

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

DiscoveringDesignConceptsforCADSketchesYuezhiYangTheUniversityofHongKongMicrosoftResearchAsiayzyang@cs.hku.hkHaoPanMicrosoftResearchAsiahaopan@microsoft.comAbstractSketchdesignconceptsarerecurringpatternsfoundinparametricCADsketches.ThoughrarelyexplicitlyformalizedbytheCADdesigners,theseconceptsare...

展开>> 收起<<

Discovering Design Concepts for CAD Sketches Yuezhi Yang The University of Hong Kong.pdf

共24页,预览5页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Discovering Design Concepts for CAD Sketches Yuezhi Yang The University of Hong Kong

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: