FRAME RATE UP-CONVERSION USING KEY POINT AGNOSTIC FREQUENCY-SELECTIVE MESH-TO-GRID RESAMPLING Viktoria Heimann Andreas Spruck and Andr e Kaup

2025-05-06 0 0 287.21KB 5 页 10玖币

侵权投诉

FRAME RATE UP-CONVERSION USING KEY POINT AGNOSTIC

FREQUENCY-SELECTIVE MESH-TO-GRID RESAMPLING

Viktoria Heimann, Andreas Spruck, and Andr´

e Kaup

Multimedia Communications and Signal Processing

Friedrich-Alexander Universit¨

at, Erlangen-N¨

urnberg

ABSTRACT

High frame rates are desired in many ﬁelds of application. As

in many cases the frame repetition rate of an already captured

video has to be increased, frame rate up-conversion (FRUC) is

of high interest. We conduct a motion compensated approach.

From two neighboring frames, the motion is estimated and the

neighboring pixels are shifted along the motion vector into the

frame to be reconstructed. For displaying, these irregularly

distributed mesh pixels have to be resampled onto regularly

spaced grid positions. We use the model-based key point ag-

nostic frequency-selective mesh-to-grid resampling (AFSMR)

for this task and show that AFSMR works best for applications

that contain irregular meshes with varying densities. AFSMR

gains up to 3.2 dB in contrast to the already high performing

frequency-selective mesh-to-grid resampling (FSMR). Addition-

ally, AFSMR increases the run time by a factor of 11 relative to

FSMR.

Index Terms—FRUC, resampling, scattered data

1. INTRODUCTION

The generation of additonal video frames is of high importance

in many applications. In entertainment industry, the demand for

additional video frames is high, e.g., for slow motion generation,

for novel view synthesis, or in frame recovery in video stream-

ing. Additionally, varying repetition rates of captured videos and

replaying devices ask for frame rate up-conversion (FRUC) [1].

Furthermore, high frame rates are also desired in video surveil-

lance and automotive applications. Hardware that can fullﬁll

these needs is too expensive in many scenarios so that the ad-

ditional frames have to be generated artiﬁcially.

There exist three categories of FRUC approaches: the recently

developed approaches using deep learning, non-motion compen-

sated, and motion compensated (MC). The recent approaches us-

ing deep learning mostly combine motion compensation and im-

age interpolation into one framework [2, 3]. Non-motion com-

pensated approaches like the projection of the temporally closest

original frame can be used as fallback solutions [1]. Also numer-

ous MC methods are proposed in literature, e.g. MC shifting, MC

averaging [4]. Additionally, more sophisticated MC approaches

are published [5, 6]. In these approaches, bidirectional motion

estimation is used for motion compensation. On the decoder side

of transmission, some scenarios demand for unidirectional mo-

[∆m, ∆n]∈ O(c−1→c+1)

F(c−1) F(c)F(c+1)

Fig. 1. The framework for FRUC applications using uni-

directional motion compensation. The vector [∆m, ∆n]∈

O(c−1→c+1) is a representant of the set of motion vectors in for-

ward direction. The frame F(c)in gray has to be reconstructed.

tion estimation and consequently, motion compensation. Fur-

thermore, motion ﬁeld estimators based on neural networks like

[7] estimate the ﬂow just in one direction as well. Hence, us-

ing unidirectional motion compensation is the most general ap-

proach of synthesizing additional video frames and increasing

the frame rate. As in these cases the number of pixels that can be

used for interpolation is only half the number of pixels than for

bidirectional motion estimation cases, it is crucial to use the best

interpolation technique available. Thus, we propose to use Key

Point Agnostic Frequency-Selective Mesh-to-Grid Resampling

(AFSMR) to solve the resampling problem. ASFMR already

showed high performance for afﬁne transforms [8]. Now, we

demonstrate the performance of AFSMR for meshes with vary-

ing densities as they result from MC.

Our framework for FRUC using unidirectional motion compen-

sation is further described in the next section. In Section 3 our

AFSMR algorithm is explained in further detail for the applica-

tion of FRUC. Subsequently, the simulation results are shown

and discussed in Section 4. Section 5 concludes this paper.

reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or

reuse of any copyrighted component of this work in other works. DOI: 10.1109/ICASSP39728.2021.9413684

arXiv:2210.10444v1 [eess.IV] 19 Oct 2022

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

FRAMERATEUP-CONVERSIONUSINGKEYPOINTAGNOSTICFREQUENCY-SELECTIVEMESH-TO-GRIDRESAMPLINGViktoriaHeimann,AndreasSpruck,andAndr´eKaupMultimediaCommunicationsandSignalProcessingFriedrich-AlexanderUniversit¨at,Erlangen-N¨urnbergABSTRACTHighframeratesaredesiredinmanyeldsofapplication.Asinmanycasestheframere...

展开>> 收起<<

FRAME RATE UP-CONVERSION USING KEY POINT AGNOSTIC FREQUENCY-SELECTIVE MESH-TO-GRID RESAMPLING Viktoria Heimann Andreas Spruck and Andr e Kaup.pdf

共5页,预览1页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

FRAME RATE UP-CONVERSION USING KEY POINT AGNOSTIC FREQUENCY-SELECTIVE MESH-TO-GRID RESAMPLING Viktoria Heimann Andreas Spruck and Andr e Kaup

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: