CFL-Net Image Forgery Localization Using Contrastive Learning Fahim Faisal Niloy Kishor Kumar Bhaumikand Simon S. Woo Center for Computational Data Sciences Independent University Bangladesh_2

2025-04-30 0 0 2.26MB 10 页 10玖币

侵权投诉

CFL-Net: Image Forgery Localization Using Contrastive Learning

Fahim Faisal Niloy†, Kishor Kumar Bhaumik‡and Simon S. Woo‡

†Center for Computational & Data Sciences, Independent University, Bangladesh

‡Computer Science and Engineering Department, Sungkyunkwan University, Suwon, South Korea

niloy9542@gmail.com, {kishor25, swoo}@g.skku.edu

Abstract

Conventional forgery localizing methods usually rely on

different forgery footprints such as JPEG artifacts, edge in-

consistency, camera noise, etc., with cross-entropy loss to

locate manipulated regions. However, these methods have

the disadvantage of over-ﬁtting and focusing on only a few

speciﬁc forgery footprints. On the other hand, real-life ma-

nipulated images are generated via a wide variety of forgery

operations and thus, leave behind a wide variety of forgery

footprints. Therefore, we need a more general approach

for image forgery localization that can work well on a va-

riety of forgery conditions. A key assumption in underly-

ing forged region localization is that there remains a differ-

ence of feature distribution between untampered and ma-

nipulated regions in each forged image sample, irrespective

of the forgery type. In this paper, we aim to leverage this dif-

ference of feature distribution to aid in image forgery local-

ization. Speciﬁcally, we use contrastive loss to learn map-

ping into a feature space where the features between un-

tampered and manipulated regions are well-separated for

each image. Also, our method has the advantage of localiz-

ing manipulated region without requiring any prior knowl-

edge or assumption about the forgery type. We demonstrate

that our work outperforms several existing methods on three

benchmark image manipulation datasets. Code is available

at https://github.com/niloy193/CFLNet

1. Introduction

Image forgery has been a serious emerging socio-

technical issue, as more advanced AI techniques have been

leveraged to create fake images. Image is a signiﬁcant

medium for information transfer. In order to produce fake

stories, academic trickery, and illegal conduct, manipulated

photographs created utilizing image editing technology are

constantly being mistaken for real ones. When a digital im-

Original Image Manipulated Image Ground Truth Mask

Figure 1: Examples of image manipulation. First two rows

show examples of image splicing and the next two rows

show examples of copy-move forgery and removal respec-

tively.

age is manipulated, we frequently assume that image foren-

sic investigations will be able to spot the tampered areas.

However, collecting differentiating features of tampered ar-

eas with various forging types (including splicing, copy-

move, removal, etc.) is still challenging and typically calls

for utilizing the special qualities of numerous tampering ar-

tifacts.

Generally, image forgery can be broadly categorized

into: splicing [12, 25], copy-move [11, 36, 35], removal

[42], enhancement [4, 9], etc. First, in image splicing,

content is copied and pasted from other source images, as

opposed to copy-move forgery, where the content is ob-

tained from the same image. On the other hand, removal

arXiv:2210.02182v1 [cs.CV] 4 Oct 2022

or inpainting techniques remove a selected region from

the image and ﬁlls the space with new pixel values esti-

mated from background [37]. Image enhancement exploits

a wide collection of local manipulations, such as sharpen-

ing, brightness adjustment, etc. Each of the broader cate-

gories can be further divided into more ﬁne-grained forgery

types. For example, Gaussian blurring or JPEG compres-

sion may be applied to the tampered region before commit-

ting splicing or copy-move forgery. Recently, more general-

purpose image forgery localization methods have been pro-

posed, which can detect or localize more than one forgery

type, such as RGB-N Net [41], Manipulation Tracing Net-

work (ManTraNet) [37], Spatial Pyramid Attention Net-

work (SPAN) [22], etc.

These general image forgery detection or localization

methods usually rely on different forgery clues or footprints

left by the forgery operation, such as JPEG artifacts [27, 1],

edge inconsistency [32, 39], noise pattern [13, 38], cam-

era model [31], EXIF inconsistency [23], etc., to detect or

localize forgery. Table 1 of [37] summarizes existing ma-

jor forgery localization methods and the forgery clues the

methods focus on. For example, [2] employs LSTM based

patch comparison to focus on edge inconsistency between

the tampered patches and authentic patches. CAT-Net [26]

leverages DCT coefﬁcients to focus on resampling clues.

However, training models to focus on speciﬁc forgery

clues has a major disadvantage. Because then, the model

can only detect forgery if that particular forgery footprint is

prominent in the forged image. This is unacceptable be-

cause, in real-life, different manipulation techniques can

leave behind wide variety of forgery clues. Thus, focus-

ing on speciﬁc forgery clues is not optimal. For example,

if a method focuses on edge inconsistency to detect forgery,

the method will not perform well on a forged image where

the boundary between untampered and manipulated region

is smooth. Again, if a method focuses on resampling fea-

tures, it will struggle to detect forgery if an image has the

same JPEG compression applied several times to both the

untampered and manipulated regions.

Another major disadvantage of existing methods is that

these methods use cross-entropy loss without additional

constraints for training. Recently, [40] stated that tra-

ditional cross-entropy based methods assume that all in-

stances within each category should be close in feature dis-

tribution. This ignores the unique information of each sam-

ple. Thus, cross-entropy loss encourages the model to ex-

tract similar features for same category. This might be help-

ful for classiﬁcation or segmentation of datasets such as Im-

agenet or Cityscapes, where objects of the same category

should have similar features. However, in the case of image

forgery localization, extracting similar features for all the

tampered regions in the dataset is not optimal as different

manipulation operations leave behind different forgery foot-

prints in the tampered regions. Hence, without additional

constraints, a common cross-entropy loss-based framework

is prone to over-ﬁtting on speciﬁc forgery patterns [28].

This is not conducive to generalization.

Taking all these limitations into consideration, we pro-

pose a novel forgery localization method named Contrastive

Forgery Localization Network or CFL-Net, based on re-

cently proposed contrastive loss [24]. Our method relies on

the general assumption in underlying forged region local-

ization that there remains a difference of feature statistics,

i.e., color, intensity, noise, etc., between untampered region

and manipulated region [22], irrespective of the forgery

type. In this paper, we focus on leveraging this difference

in the feature space to aid in image forgery localization via

contrastive loss. Speciﬁcally, our model learns mapping

into a feature space where the features between untampered

and manipulated regions are well-separated and dispersed

for each image. Thus, our method does not focus on spe-

ciﬁc forgery clues. Also, we calculate the contrastive loss

for each sample. Hence, our method treats the forgery clues

of each sample differently, which helps in generalization.

Our main contributions are summarized as follows:

• We propose a novel image forgery localization method

called CFL-Net. Our method leverages the difference

of feature distribution between untampered and manip-

ulated regions of each image sample and does not fo-

cus on speciﬁc forgery footprints. Hence, our method

is more well-suited to detect real-life forgery.

• We address the problem of using cross-entropy loss

without any constraints for general purpose image

forgery localization. We incorporate contrastive loss

and especially tailor it towards solving this problem.

• We perform extensive experiments on benchmark ma-

nipulation datasets to show that our method out-

performs several existing image forgery localization

methods.

2. Related Works

2.1. Image Forgery Localization

Image forgery methods are concerned with forgery clas-

siﬁcation or localization. Classiﬁcation is basically predict-

ing whether an image is forged or non-forged, whereas,

forgery localization is concerned with locating the forged

region as well. The latter is a segmentation task.

In pre deep learning era, methods used hand-crafted fea-

tures such as local noise analysis [16, 10], CFA artifacts

[15], JPEG compression [5] etc. Recent works usually

use deep learning based methods in conjunction with these

forgery traces to localize forged regions. Bappy et al. [2]

exploit the edge inconsistency trace using LSTM to local-

ize forgery. The work is later improved in [3], where the

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

CFL-Net:ImageForgeryLocalizationUsingContrastiveLearningFahimFaisalNiloy†,KishorKumarBhaumik‡andSimonS.Woo‡†CenterforComputational&DataSciences,IndependentUniversity,Bangladesh‡ComputerScienceandEngineeringDepartment,SungkyunkwanUniversity,Suwon,SouthKoreaniloy9542@gmail.com,{kishor25,swoo}@g.skku.e...

展开>> 收起<<

CFL-Net Image Forgery Localization Using Contrastive Learning Fahim Faisal Niloy Kishor Kumar Bhaumikand Simon S. Woo Center for Computational Data Sciences Independent University Bangladesh_2.pdf

共10页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

CFL-Net Image Forgery Localization Using Contrastive Learning Fahim Faisal Niloy Kishor Kumar Bhaumikand Simon S. Woo Center for Computational Data Sciences Independent University Bangladesh_2

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: