DeepMend Learning Occupancy Functions to Represent Shape for Repair Nikolas Lamb Sean Banerjee and Natasha Kholgade Banerjee

2025-04-26 0 0 5.36MB 18 页 10玖币

侵权投诉

DeepMend: Learning Occupancy Functions to

Represent Shape for Repair

Nikolas Lamb , Sean Banerjee , and Natasha Kholgade Banerjee

Clarkson University, Potsdam, NY 13699, USA

{lambne,sbanerje,nbanerje}@clarkson.edu

Abstract. We present DeepMend, a novel approach to reconstruct resto-

rations to fractured shapes using learned occupancy functions. Existing

shape repair approaches predict low-resolution voxelized restorations, or

require symmetries or access to a pre-existing complete oracle. We repre-

sent the occupancy of a fractured shape as the conjunction of the occu-

pancy of an underlying complete shape and the fracture surface, which

we model as functions of latent codes using neural networks. Given occu-

pancy samples from an input fractured shape, we estimate latent codes

using an inference loss augmented with novel penalty terms that avoid

empty or voluminous restorations. We use inferred codes to reconstruct

the restoration shape. We show results with simulated fractures on syn-

thetic and real-world scanned objects, and with scanned real fractured

mugs. Compared to the existing voxel approach and two baseline meth-

ods, our work shows state-of-the-art results in accuracy and avoiding

restoration artifacts over non-fracture regions of the fractured shape.

Keywords: Learned Occupancy, Shape Representation, Repair, Frac-

ture, Implicit Surface, Neural Networks

1 Introduction

Automated restoration of fractured shapes is an important area of study, with ap-

plications in consumer waste reduction, commercial recycling, cultural heritage

object restoration, medical ﬁelds such as orthopedics and dentistry, and robot-

driven repair. Despite its wide application, automated repair of fractured shapes

has received little attention. Most current automated techniques use symmetries

to complete fractured shapes [16,34]. These techniques do not generalize to ob-

jects with non-symmetrical damage. The only existing generalizable approach

for repair operates in voxel space [21] and produces low-ﬁdelity restorations.

In this work, we present DeepMend, a novel deep learning approach to gener-

ate high-ﬁdelity restoration shapes given an input fractured shape. Our approach

is inspired by work on learning functions for quantities such as signed distance

ﬁeld (SDF) or occupancy that implicitly represent the shape surface over the

continuous 3D domain [6, 13, 20, 35, 42, 48, 48, 53, 58]. These approaches perform

partial shape completion by using the learned function to infer a latent code us-

ing ﬁeld value samples from a partial shape, and to compute the complete shape

This preprint has not undergone any post-submission improvements or corrections.

The Version of Record of this contribution is published in Proceedings of the 17th

European Conference on Computer Vision (ECCV 2022).

arXiv:2210.05728v1 [cs.CV] 11 Oct 2022

2 N. Lamb et al.

Input Fractured Shapes, Predicted Restoration Shapes in Red, and Ground Truth Restoration Shapes in Red for Various Objects

Break Code

Complete

Code

Input Fractured

Shape

Predicted

Complete Shape

Predicted Restoration

Shape in Red

Ground Truth Restoration

Shape in Red

Predicted Break

Surface

Break

Region

Fig. 1. Given a fractured shape, our approach infers latent codes for an underlying

complete shape and a break surface. We use the codes to generate a restoration shape

that repairs the input fracture shape.

values from the latent code. Diﬀerent from partial shape completion, DeepMend

addresses the challenge that, unlike a partial shape that is a subset of a complete

shape, the fractured shape contains a novel break region missing in a complete

shape, as shown in Figure 1. To restore fractured shapes, our main contribution

is a novel representation that represents the occupancy of a fractured shape as

the logical conjunction of occupancy values for a complete shape and a break

surface. We use T-norms [18] to relax the logical conjunction into arithmetic op-

erations. We represent the complete and break surface occupancy as functions

parametrized on latent codes and modeled using deep neural networks.

Given an input fractured shape, we use the learned functions to automatically

estimate latent codes for the complete shape and break surface after sampling oc-

cupancy values from the fractured shape. Our second contribution is to augment

the inference loss for latent code estimation with two penalty terms—(i) a non-

empty restoration term that penalizes the mean restoration occupancy against

being zero to avoid empty restorations, and (ii) a proximity term that encourages

the mean distance between the complete and restoration occupancy to be low

to prevent voluminous restorations. Our work naturally yields the restoration

occupancy as the conjunction of the complete occupancy and the negation of

the break occupancy, enabling its reconstruction using Marching Cubes [30].

We train and test our approach on synthetically fractured meshes from 8

classes from the ShapeNet [5] dataset, and on the Google Scanned Objects

dataset [39] which contains 1,032 scanned real-world objects. We use ShapeNet-

trained networks to restore synthetically fractured meshes from the QP Cul-

tural Heritage dataset [23], and to generate restorations for physically fractured

and scanned real-world mugs. We compare our work to 3D-ORGAN [21], the

only existing automated fracture restoration approach, and to two baselines. We

show state-of-the-art results in overall accuracy and avoiding inaccurate arti-

facts over non-fracture regions. Our code is available at https://github.com/

Terascale-All-sensing-Research-Studio/DeepMend.

DeepMend: Learning Occupancy Functions to Represent Shape for Repair 3

2 Related Work

Restoration of Fractured Shapes. Most existing approaches to generate

restoration shapes from fractured shapes rely on shape symmetry [16,34]. They

restore shapes by reﬂecting non-fractured regions of the shape onto fractured

regions and computing the subtraction. These approaches fail to restore asym-

metrical shapes or shapes that have non-symmetric fractures. Lamb et al. [24]

perform repair without relying on symmetries. However, they require that the

complete counterpart be provided as input alongside the fractured shape. The

complete shape may not always be available, e.g., in the case of a rare object.

Our work only requires the fractured shape as input. 3D-ORGAN [21] performs

shape restoration in voxel space by using a voxelized representation of an in-

put fractured shape as input to a generative adversarial network. 3D-ORGAN

operates at a resolution of 32×32×32, which is insuﬃcient to accurately repre-

sent the geometric complexity of the fracture region. Scaling 3D-ORGAN to a

voxel resolution necessary to represent fracture is impractical at current dataset

volumes and hardware. DeepMend overcomes the challenges of 3D-ORGAN by

using networks that represent point samples of the occupancy function.

Completion of Partial Shapes. Though not directly related to our work,

a large body of prior work focuses on completing shapes from partial shape

representations, e.g. depth maps or color images. Recent approaches hypothe-

size complete shapes from partial shapes using deep networks. Approaches that

use point clouds as input [1, 10, 19, 29, 33, 40, 44, 57] lack an intrinsic surface

representation. Some approaches predict 3D meshes [17, 56] to incorporate sur-

faces. These approaches are limited in the complexity of meshes reliably pre-

dicted [32], and cannot represent arbitrary topologies. Most approaches using

voxels [3,41,43,49] struggle to predict high-resolution outputs while being compu-

tationally tractable. Some voxel approaches address computational ineﬃciency

by employing hierarchical models [8, 9] or sparse convolutions [8, 55]. However,

voxel approaches pre-discretize the domain, making it challenging to use them

to represent arbitrarily ﬁne resolutions needed for geometric detail, especially

for the problem of fracture surface representation considered in this work.

A large body of recent work focuses on using neural networks to represent

point samples of values that implicitly deﬁne surfaces, e.g., occupancy [6, 7, 14,

22,26, 28, 32,36, 37, 46, 52,53], signed distance ﬁeld (SDF) [4, 13, 20,27, 31, 35, 42,

48,50,51,54,58], unsigned distance ﬁeld [47], or level sets [15]. These approaches

show high reconstruction ﬁdelity due to their ability to represent the continu-

ous domain of points, while remaining computationally tractable. In contrast to

traditional encoder-decoder architectures, approaches based on the autodecoder

introduced by DeepSDF [35] use maximum a posteriori estimation to obtain a

latent code for a give shape. The approach enables reconstruction of a complete

shape using a latent code estimated using incomplete shape observations. Later

approaches provide improvements to the autodecoder by using meta-learning

and post-training optimization [42, 54], and by learning increasingly complex

shape representations during training [11], deformation of implicit shape tem-

plates [58], and reconstruction of shapes at multiple resolutions [20].

4 N. Lamb et al.

A potential approach for fractured shape restoration is to convert the frac-

tured shape into a partial shape by removing the fracture surface, perform shape

completion, and subtract the fractured shape from the completed shape to ob-

tain the restoration. We demonstrate in Section 5 that subtraction approaches

yield surface artifacts that extend over the non-fracture regions of the fractured

shape. Our approach mitigates non-fracture artifacts by learning the interplay

between the complete shape and break surface.

3 Representing Fractured Shapes

We represent the complete, fractured, and restoration shapes as point sets C,

F, and R. For S ∈ {C, F, R}the occupancy oS(x)∈ {0,1}of a point xis 1

if xis inside the shape, and 0 if it is on the boundary or outside the shape.

The original shapes are closed surfaces. However, we exclude boundary points

from the deﬁnitions of the sets C,F, and Rto ensure that a point does not

simultaneously belong to two sets, e.g., Fand R. Exclusion of boundary points

makes the sets C,F, and Ropen and bounded. We deﬁne the break surface as

a 2D surface that intersects the fracture region of F. Points on the side of the

break surface corresponding to the fractured shape receive an occupancy of 1.

Points on the side corresponding to the restoration shape have an occupancy

of 0. We use the open unbounded set B, termed the ‘break set’ to represent

the set of points with an occupancy oB(x) of 1. In principle, the break surface

is inﬁnite. In practice, we limit the region containing the shape and break sets

to be a cube of ﬁnite length to make point sampling for network training and

inference tractable1.

As shown in Figure 2(a), we represent the fractured shape set as the inter-

section of the sets for the complete shape and the break set, i.e, as F=C∩B.

The set relationship implies that for a point x, occupancy oF(x) is the logical

conjunction of the occupancy values oC(x) and oB(x) of the complete shape

Cand break set B, i.e., oF(x) = oC(x)∧oB(x). We represent the restoration

shape as the intersection of the complete shape and the complement of the break

set, i.e, as R=C∩B′. The relationship implies that occupancy oR(x) of the

restoration Ris expressed as the logical conjunction of oC(x) with the nega-

tion of oB(x), i.e., oR(x) = oC(x)∧¬oB(x). The logical relationships are shown

in Figure 2(b). To use the expressions in neural networks, we relax the logical

relationships using the product T-norm [18], as

oF(x) = oC(x)oB(x) and (1)

oR(x) = oC(x)(1 −oB(x)).(2)

We represent the occupancy functions for the complete shape Cand break set B

respectively using neural networks fΘand gΦ, such that oC(x) = fΘ(zC,x) and

oB(x) = gΦ(zB,x). Θand Φare the network weights, zC∈Rpis a latent code

of size pcorresponding to the complete shape, and zB∈Rqis a latent code of

1Hereafter, we drop ‘set’ from references to C,F, and R, and refer to them as shapes.

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

DeepMend:LearningOccupancyFunctionstoRepresentShapeforRepairNikolasLamb,SeanBanerjee,andNatashaKholgadeBanerjeeClarksonUniversity,Potsdam,NY13699,USA{lambne,sbanerje,nbanerje}@clarkson.eduAbstract.WepresentDeepMend,anovelapproachtoreconstructresto-rationstofracturedshapesusinglearnedoccupancyfunctio...

展开>> 收起<<

DeepMend Learning Occupancy Functions to Represent Shape for Repair Nikolas Lamb Sean Banerjee and Natasha Kholgade Banerjee.pdf

共18页,预览4页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

DeepMend Learning Occupancy Functions to Represent Shape for Repair Nikolas Lamb Sean Banerjee and Natasha Kholgade Banerjee

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: