Deep Learning-based Facial Appearance Simulation Driven by Surgically Planned Craniomaxillofacial Bony Movement

2025-05-06 0 0 3.7MB 12 页 10玖币

侵权投诉

Deep Learning-based Facial Appearance

Simulation Driven by Surgically Planned

Craniomaxillofacial Bony Movement

Xi Fang1∗, Daeseung Kim2∗, Xuanang Xu1, Tianshu Kuang2, Hannah H.

Deng2, Joshua C. Barber2, Nathan Lampen1, Jaime Gateno2, Michael A.K.

Liebschner3, James J. Xia2( ), and Pingkun Yan1( )

1Department of Biomedical Engineering and Center for Biotechnology and

Interdisciplinary Studies, Rensselaer Polytechnic Institute, Troy, NY 12180, USA

2Department of Oral and Maxillofacial Surgery, Houston Methodist Research

Institute, Houston, TX 77030, USA

3Department of Neurosurgery, Baylor College of Medicine, Houston, TX 77030, USA

Abstract. Simulating facial appearance change following bony move-

ment is a critical step in orthognathic surgical planning for patients with

jaw deformities. Conventional biomechanics-based methods such as the

ﬁnite-element method (FEM) are labor intensive and computationally in-

eﬃcient. Deep learning-based approaches can be promising alternatives

due to their high computational eﬃciency and strong modeling capabil-

ity. However, the existing deep learning-based method ignores the physi-

cal correspondence between facial soft tissue and bony segments and thus

is signiﬁcantly less accurate compared to FEM. In this work, we propose

an Attentive Correspondence assisted Movement Transformation net-

work (ACMT-Net) to estimate the facial appearance by transforming

the bony movement to facial soft tissue through a point-to-point atten-

tive correspondence matrix. Experimental results on patients with jaw

deformity show that our proposed method can achieve comparable facial

change prediction accuracy compared with the state-of-the-art FEM-

based approach with signiﬁcantly improved computational eﬃciency.

Keywords: Deep Learning ·Surgical Planning ·Simulation ·Corre-

spondence assisted movement transformation ·Cross point-set Attention

1 Introduction

Orthognathic surgery is a bony surgical procedure (called “osteotomy”) to cor-

rect jaw deformities. During orthognathic surgery, the maxilla and the mandible

are osteotomized into multiple segments, which are then individually moved to

a desired (normalized) position. While orthognathic surgery does not directly

?X. Fang and D. Kim contributed equally to this paper.

?? J.J. Xia (JXia@houstonmethodist.org) and P. Yan (yanp2@rpi.edu) are co-

corresponding authors.

arXiv:2210.01685v1 [cs.CV] 4 Oct 2022

2 X. Fang et al.

operate on facial soft tissue, the facial appearance automatically changes fol-

lowing the bony movement [13]. Surgeons now can accurately plan the bony

movement using computer-aided surgical simulation (CASS) technology in their

daily practice [17]. However, accurate and eﬃcient prediction of the facial ap-

pearance change following bony movement is still a challenging task due to the

complicated nonlinear relationship between facial soft tissues and underlying

bones [3].

Finite-element method (FEM) is currently acknowledged as the most physically-

relevant and accurate method for facial change prediction. However, despite

the eﬀorts to accelerate FEM [1, 2], FEM is still time-consuming and labor-

intensive because it requires heavy computation and manual mesh modeling

to achieve clinically acceptable accuracy [3]. In addition, surgical planning for

orthognathic surgery often requires multiple times of revisions to achieve ideal

surgical outcomes, therefore preventing facial change prediction using FEM from

being adopted in daily clinical setting [5].

Deep learning-based approaches have been recently proposed to automate

and accelerate the surgical simulation. Li et al. [6] proposed a spatial trans-

former network based on the PointNet [11] to predict tooth displacement for

malocclusion treatment planning. Xiao et al. [18] developed a self-supervised

deep learning framework to estimate normalized facial bony models to guide or-

thognathic surgical planning. However, these studies are not applicable to facial

change simulation because they only allows single point set as input whereas fa-

cial change simulations require two point sets, i.e., bony and facial surface points.

Especially, in-depth modeling of the correlation between bony and facial surfaces

is the key factor for accurate facial change prediction using deep learning tech-

nology. Ma et al. [9] proposed a facial appearance change simulation network,

FC-Net, that embedded the bony-facial relationship into facial appearance simu-

lation. FC-Net takes both bony and facial point sets as input to jointly infer the

facial change following the bony movement. However, instead of explicitly estab-

lishing spatial correspondence between bony and facial point sets, the movement

vectors of all bony segments in FC-Net are represented by a single global feature

vector. Such a global feature vector ignores local spatial correspondence and may

lead to compromised accuracy in facial change simulation.

In this study, we hypothesize that establishing point-to-point correspon-

dence between the bony and facial point sets can accurately transfer the bony

movement to the facial points and in turn signiﬁcantly improve the postopera-

tive facial change prediction. To test our hypothesis, we propose an Attentive

Correspondence assisted Movement Transformation network (ACMT-Net) that

equipped with a novel cross point-set attention (CPSA) module to explicitly

model the spatial correspondence between facial soft tissue and bony segments

by computing a point-to-point attentive correspondence matrix between the two

point sets. Speciﬁcally, we ﬁrst utilize a pair of PointNet++ networks [12] to ex-

tract the features from the input bony and facial point sets, respectively. Then,

the extracted features are fed to the CPSA module to estimate the point-to-point

correspondence between each bony-facial point pair. Finally, the estimated at-

Deep Learning-based Facial Appearance Simulation 3

PointNet++

Mesh

Generation

Feature Extraction Movement Transformation

PointNet++

Planning

Pre-facial model

Pre-bony model

Post-bony model

𝑁!×64

𝑁"×64

Normalize

𝑁!×6

𝐹

!"#$%

(𝑁"×𝐷)

𝜃:

𝜑:

𝑃

!"#$%

(𝑁"×3)

𝑃

&"#$%

(𝑁!×3)

𝑉

(𝑁!×3)

𝑃

!"#'() ′

(𝑁"×3)

Cross Point-set Attention

(CPSA)

𝐹

&"#$%

(𝑁!×𝐷)

𝑅

(𝑁"× 𝑁!)

𝑉

(𝑁"×3)

Estimated

Post-facial model

Conv 1D

Element-wise sum

Concatenation

Matrix multiplication

Sigmoid activation

Fig. 1. Scheme of the proposed Attentive Correspondence assisted Movement Trans-

formation network (ACMT-Net) for facial change simulation.

tention matrix is used to transfer the bony movement to the preoperative facial

surface to simulate postoperative facial change.

The contributions of our work are two-fold. 1) From the technical perspective,

an ACMT-Net with a novel CPSA module is developed to estimate the change

of one point set driven by the movement of another point set. The network

leverages the local movement vector information by explicitly establishing the

spatial correspondence between two point sets 2) From the clinical perspective,

the proposed ACMT-Net can achieve a comparable accuracy of the state-of-the-

art FEM simulation method, while substantially reducing computational time

during the surgical planning.

2 Method

ACMT-Net predicts postoperative facial model (three-dimensional (3D) surface)

based on preoperative facial and bony model and planned postoperative bony

model. ACMT-Net is composed of two major components: 1) point-wise feature

extraction and 2) point-wise facial movement prediction (Fig. 1). In the ﬁrst com-

ponent, pre-facial point set PF−pre, pre-bony point set PB−pre, and post-bony

point set PB−post are subsampled from the pre- and post- facial/bony models for

computational eﬃciency. Then the pre-facial/bony point sets (PF−pre,PB−pre)

are fed into a pair of PointNet++ networks to extract semantical and topological

features FF−pre and FB−pre, respectively. In the second component,FF−pre and

FB−pre are fed into the CPSA module to estimate the point-to-point correspon-

dence between facial and bony points. Sequentially, the estimated point-to-point

correspondence is combined with pre bony points PB−pre and bony points move-

ment VBto estimate point-wise facial movement VF(i.e., the displacement from

the pre-facial points PF−pre to the predicted post-facial points PF−post0). Fi-

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

DeepLearning-basedFacialAppearanceSimulationDrivenbySurgicallyPlannedCraniomaxillofacialBonyMovementXiFang1,DaeseungKim2,XuanangXu1,TianshuKuang2,HannahH.Deng2,JoshuaC.Barber2,NathanLampen1,JaimeGateno2,MichaelA.K.Liebschner3,JamesJ.Xia2(),andPingkunYan1()1DepartmentofBiomedicalEngineeringandCente...

展开>> 收起<<

Deep Learning-based Facial Appearance Simulation Driven by Surgically Planned Craniomaxillofacial Bony Movement.pdf

共12页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Deep Learning-based Facial Appearance Simulation Driven by Surgically Planned Craniomaxillofacial Bony Movement

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: