Self-Aligned Concave Curve Illumination Enhancement for Unsupervised Adaptation

2025-05-03 0 0 5.85MB 10 页 10玖币

侵权投诉

Self-Aligned Concave Curve: Illumination Enhancement for

Unsupervised Adaptation

Wenjing Wang

Peking University

Beijing, China

daooshee@pku.edu.cn

Zhengbo Xu

Peking University

Beijing, China

icey.x@pku.edu.cn

Haofeng Huang

Peking University

Beijing, China

hhf@pku.edu.cn

Jiaying Liu∗

Peking University

Beijing, China

liujiaying@pku.edu.cn

ABSTRACT

Low light conditions not only degrade human visual experience,

but also reduce the performance of downstream machine analytics.

Although many works have been designed for low-light enhance-

ment or domain adaptive machine analytics, the former considers

less on high-level vision, while the latter neglects the potential of

image-level signal adjustment. How to restore underexposed im-

ages/videos from the perspective of machine vision has long been

overlooked. In this paper, we are the rst to propose a learnable

illumination enhancement model for high-level vision. Inspired by

real camera response functions, we assume that the illumination

enhancement function should be a concave curve, and propose to

satisfy this concavity through discrete integral. With the intention

of adapting illumination from the perspective of machine vision

without task-specic annotated data, we design an asymmetric

cross-domain self-supervised training strategy. Our model archi-

tecture and training designs mutually benet each other, forming

a powerful unsupervised normal-to-low light adaptation frame-

work. Comprehensive experiments demonstrate that our method

surpasses existing low-light enhancement and adaptation methods

and shows superior generalization on various low-light vision tasks,

including classication, detection, action recognition, and optical

ow estimation. All of our data, code, and results will be available

online upon publication of the paper.

CCS CONCEPTS

•Computing methodologies →Image processing

;

Computer

vision problems.

∗

Corresponding Author. This work was supported by the National Natural Science

Foundation of China under Contract No.62172020, and a research achievement of Key

Laboratory of Science, Techonology and Standard in Press Industry (Key Laboratory

of Intelligent Press Media Technology).

Permission to make digital or hard copies of all or part of this work for personal or

classroom use is granted without fee provided that copies are not made or distributed

for prot or commercial advantage and that copies bear this notice and the full citation

on the rst page. Copyrights for components of this work owned by others than ACM

must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,

to post on servers or to redistribute to lists, requires prior specic permission and/or a

fee. Request permissions from permissions@acm.org.

MM ’22, October 10–14, 2022, Lisboa, Portugal

ACM ISBN 978-1-4503-9203-7/22/10. . . $15.00

https://doi.org/10.1145/3503161.3547991

KEYWORDS

Low-light, domain adaptation, low-level vision, high-level vision

1 INTRODUCTION

Insucient lighting is a common kind of image degradation, caused

by dark environments, corrupted equipment, or improper shooting

settings. It may degrade the visual quality of images, causing low

visibility, lost details, and aesthetic distortion. Besides, with the rise

of machine learning, visual analytics has been playing an increas-

ingly critical role in many applications. Low light can also pose

threats to machine analytics, presenting challenges to high-level vi-

sion tasks in the low-light condition, such as nighttime autonomous

driving and surveillance video analysis.

The research of restoring low-light images has attracted wide

attention. From early manually designed algorithms [

] to recent

data-driven deep models [

], a great number of works have eec-

tively improved human visual quality of low-light images. However,

most existing low-light enhancement methods do not take machine

vision into account. Some methods introduce noise removal [

] or

detail reconstruction [

], which enhances visual quality but might

mislead high-level analytics models. Some other methods take se-

mantic perception into consideration [

], but they still focus on

human vision and have unsatisfactory performance on downstream

high-level vision tasks.

In recent years, to promote the development of machine analytics

in low-light scenarios, many datasets have been built [

], giv-

ing birth to a series of machine vision models tailored for underlit

environments. With a large amount of training data, many methods

improve normal light frameworks by incorporating the proper-

ties of insucient lighting conditions [

]. Some works further ex-

plore the task of adapting normal light models to low-light without

task-specic annotation, which is more universal and application-

driven [

]. However, existing normal-to-low light adapta-

tion algorithms either rely on multiple sources [

], adopt trouble-

some multi-stage and multi-level processes [

], or fail in darker

cases [

]. Moreover, most adaptive methods concentrate on the

high-dimensional features of machine analytics models, but neglect

the characteristics of input images themselves.

Dierent from existing normal-to-low light adaptation methods,

we make full use of the potential of illumination adjustment. We

rst build an illumination enhancement model which can maximize

the model’s abilities while being easy to learn. Inspired by camera

arXiv:2210.03792v1 [cs.CV] 7 Oct 2022

MM ’22, October 10–14, 2022, Lisboa, Portugal Wenjing Wang, Zhengbo Xu, Haofeng Huang, & Jiaying Liu

response functions, we design a new model paradigm: deep concave

curve, which can determine the new pixel value in the enhanced

result with a high degree of freedom. To eectively satisfy concavity,

we propose to rst predict a non-positive second derivative, then

apply discrete integral implemented by convolutions. To train this

model towards unsupervised adaptation, we design asymmetric self-

supervised alignment. On the normal light side, we learn decision

heads with a self-supervised pretext task. Then on the low-light

side, we x the decision heads and let our model improve the pretext

task performance through enhancing the input image. In this way,

even without annotated data, our model can learn how to make

the machine analytics model better perceive the enhanced low-

light image. To make full use of image information and provide

good guidance for illumination enhancement, we propose a new

rotated jigsaw permutation task. Experiments show that our model

architecture and training design are compatible with each other. On

one hand, our self-learned strategy can better restore illumination

compared with other feature adaptation strategies; on the other

hand, our deep concave curve can best maximize the potential of

self-learned illumination alignment.

The proposed illumination enhancement model, self-aligned con-

cave curve (SACC), can serve as a powerful tool for unsupervised

low-light adaptation. Although SACC does not require normal or

low-light annotations and does not even adjust the downstream

model, it achieves superior performance on a variety of low-light

vision tasks. To further deal with noises and semantic domain gaps,

we propose to adapt downstream analytics models by pseudo la-

beling. Finally, we build an adaptation framework SACC+, which

is concise and easy to implement but can outperform existing low-

light enhancement and adaptation methods by a large margin.

In summary, our contributions are threefold:

•

We are the rst to propose a learnable pure illumination

enhancement model for high-level vision. Inspired by cam-

era response functions, we design a deep concave curve.

Through discrete integral, the concavity constraint can be

satised through the model architecture itself.

•

Towards unsupervised normal-to-low light adaptation, we

design an asymmetric cross-domain self-supervised training

strategy. Guided by the rotated jigsaw permutation pretext

task, our curve can adjust illumination from the perspective

of machine vision.

•

To verify the eectiveness of our method, we explore various

high-level vision tasks, including classication, detection,

action recognition, and optical ow estimation. Experiments

demonstrate our superiority over both low-light enhance-

ment and adaptation state-of-the-art methods.

2 RELATED WORKS

Low-light Enhancement.

Early methods manually design illu-

mination models and enhancement strategies. In the Retinex the-

ory [

], images are decomposed into reectance (albedo) and shad-

ing (illumination). On this basis, Retinex-based methods [

]

rst decompose images and then either separately or simultane-

ously process the two components. Histogram equalization and its

variants [48] instead redistribute the intensities on the histogram.

Recent methods are mainly based on deep learning. Some mod-

els mimic the Retinex decomposition process [

]. RUAS [

]

unrolls the optimization process of Retinex-inspired models and

searches desired network architectures. EnlightenGAN [

] intro-

duces adversarial learning. Zero-DCE [

] designs a curve-based

low-light enhancement model and learns in a zero-reference way.

Some methods also target RAW images [

], videos [

], and in-

troduce extra light sources [

]. Interested readers may refer to

[37] and [33] for comprehensive surveys.

Existing low-light enhancement methods disregard downstream

machine learning tasks. In comparison, our model targets high-level

vision and greatly benets machine vision in low-light scenarios.

High-Level Vision in Low-light scenarios.

With an increasing

demand for autonomous driving and surveillance analysis, low-light

high-level vision has attracted ever-higher attention in recent years.

For dark object detection, Sasagawa et al. [

] merged pretrained

models in dierent domains with glue layers and generative knowl-

edge distillation. MAET [

] learns through jointly decoding de-

grading transformations and detection predictions. HLA-Face [

]

adopts a joint pixel-level and feature-level adaptation framework.

For nighttime semantic segmentation, DANNet [

] employs ad-

versarial training to adapt models in one stage without additional

day-night image transferring. For general tasks, CIConv [

] de-

signed a color invariant representation. Some works also focus on

tasks of image retrieval [

], depth estimation [

], and match-

ing [52] in low-light conditions.

Despite all these progress on high-level vision in low-light sce-

narios, many methods rely on low-light annotations, which are

neither robust nor exible enough. Existing unsupervised adapta-

tion methods focus on feature migration and ignore the importance

of pixel-level adjustment. Based on illumination enhancement, we

propose a new method for low-light adaptation that outperforms

existing methods by a wide margin.

3 DEEP CONCAVE CURVE

In this section, we introduce the motivation and detailed architec-

ture of our illumination enhancement model.

3.1 From CRF to Concave Curve

Digital photographic cameras use camera response functions (CRFs)

when mapping irradiance to intensities. Although scene illumina-

tion changes linearly on the irradiance level, to t the logarithmic

perception of human vision, cameras employ non-linear CRFs, mak-

ing illumination adjustment complicated on the intensity level. To

utilize the linearity of irradiance, some low-light enhancement

methods [

] transform intensities to irradiance, adjust the ir-

radiance, and then map irradiance back to intensities. However,

back-and-forth irradiance

↔

intensity mapping is inconvenient

and dicult to introduce high-level machine vision guidance.

We propose to simplify the above complex pipeline into one

single intensity-level adjustment, which is denoted by

𝑔

. We rst

analyze what form

𝑔

should take. Ignoring spatial variations like

lens fall-o [

], vignetting, and signal-dependent noise, CRF can be

assumed to be the same for each pixel in an image. Accordingly, we

set 𝑔to be spatially shared. Second, to follow the numerical range

of pixels and preserve order,

𝑔

should pass

(

)

(

)

, and in-

crease monotonically. Additionally, although pixels are discrete, we

want

𝑔

to appear roughly continuous, i.e., like a curve. Despite the

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

Self-AlignedConcaveCurve:IlluminationEnhancementforUnsupervisedAdaptationWenjingWangPekingUniversityBeijing,Chinadaooshee@pku.edu.cnZhengboXuPekingUniversityBeijing,Chinaicey.x@pku.edu.cnHaofengHuangPekingUniversityBeijing,Chinahhf@pku.edu.cnJiayingLiu∗PekingUniversityBeijing,Chinaliujiaying@pku.edu...

展开>> 收起<<

Self-Aligned Concave Curve Illumination Enhancement for Unsupervised Adaptation.pdf

共10页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Self-Aligned Concave Curve Illumination Enhancement for Unsupervised Adaptation

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: