Facial De-occlusion Network for Virtual Telepresence Systems Surabhi Gupta Ashwath Shetty Avinash Sharma International Institute of Information Technology Hyderabad

2025-05-06 0 0 5.45MB 5 页 10玖币

侵权投诉

Facial De-occlusion Network for Virtual Telepresence Systems

Surabhi Gupta Ashwath Shetty Avinash Sharma

International Institute of Information Technology Hyderabad

{surabhi.gupta, ashwath.shetty}@research.iiit.ac.in

asharma@iiit.ac.in

Abstract

To see what is not in the image is one of the broader mis-

sions of computer vision. Technology to inpaint images has

made signiﬁcant progress with the coming of deep learning.

This paper proposes a method to tackle occlusion speciﬁc

to human faces. Virtual presence is a promising direction

in communication and recreation for the future. However,

Virtual Reality (VR) headsets occlude a signiﬁcant portion

of the face, hindering the photo-realistic appearance of the

face in the virtual world. State-of-the-art image inpainting

methods for de-occluding the eye region does not give us-

able results. To this end, we propose a working solution

that gives usable results to tackle this problem enabling the

use of the real-time photo-realistic de-occluded face of the

user in VR settings.

This is an extended abstract of our CRV 2022 poster pa-

per [4].

1. Introduction

Face is a principle in any dialogue. Modern technol-

ogy in the face-to-face conversation has gone from video

to virtual presence over a distance. With the coming of Vir-

tual Reality (VR), which promises to provide realism, most

technology has been developed around creating a virtual

avatar. Nonetheless, using an avatar reduces the immer-

sion and the sense of self-bringing in an uncanny valley. [6]

shows that increasing realism in the avatar does not improve

the experience. The very construct of a VR headset restricts

the use of the face, hindering the real-time broadcast of the

user’s face.

VR environment improves immersion and user experi-

ence [11], but the occlusion of the face brings down this po-

tential. Existing methods of reconstructing the face while

using a VR device do not give usable results. The primary

research challenge with the face completion/inpainting task

comes from its ill-posed nature as a signiﬁcant part of the

face is occluded by HMD. Learning a single common face

Input

(Occluded)

Reconstructed

(Un-occluded)

Groundtruth

(Un-occluded)

Figure 1. The proposed approach reconstruct unoccluded face im-

age with high-quality even in presence of apparels.

de-occlusion network with the capability to hallucinate di-

verse expressions in varying appearances and head poses

across a large set of human faces is difﬁcult to achieve. It

is due to the broad space of facial geometry and appearance

as well as the highly subjective way of articulating expres-

sions/emotions across individuals [1]. This motivates us

to devise a person-speciﬁc model that gives usable results

with high visual appeal and enables the use of the real-time

photo-realistic inpainted face of the user in VR.

Traditional inpainting methods [14–16] do not general-

ize to even slight non-frontal head poses and suffer from

a loss of identity. Methods using an additional reference

image to preserve identity also fail to scale beyond frontal

head pose [9,12]. Elgharib et al. shows that training a

person-speciﬁc model is an effective way to model high-

frequency details in generating frontal views from egocen-

tric frames [3].

As proposed, our method de-occludes face images to en-

able virtual presence without using 3D models. Previous

methods that predict view conditioned texture and mesh ge-

ometry from HMD occluded input need calibrated multi-

arXiv:2210.12622v1 [cs.CV] 23 Oct 2022

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

FacialDe-occlusionNetworkforVirtualTelepresenceSystemsSurabhiGuptaAshwathShettyAvinashSharmaInternationalInstituteofInformationTechnologyHyderabadfsurabhi.gupta,ashwath.shettyg@research.iiit.ac.inasharma@iiit.ac.inAbstractToseewhatisnotintheimageisoneofthebroadermis-sionsofcomputervision.Technologyt...

展开>> 收起<<

Facial De-occlusion Network for Virtual Telepresence Systems Surabhi Gupta Ashwath Shetty Avinash Sharma International Institute of Information Technology Hyderabad.pdf

共5页,预览1页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Facial De-occlusion Network for Virtual Telepresence Systems Surabhi Gupta Ashwath Shetty Avinash Sharma International Institute of Information Technology Hyderabad

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: