Leveraging Computer Vision Application in Visual Arts A Case Study on the Use of Residual Neural Network to Classify and

2025-05-02 0 0 4.08MB 14 页 10玖币

侵权投诉

Leveraging Computer Vision Application in

Visual Arts: A Case Study on the Use of

Residual Neural Network to Classify and

Analyze Baroque Paintings

Daniel Kvak∗

Faculty of Arts

Masaryk University

Brno, Czech Republic

ORCID: 0000-0001-7808-7773

October 28, 2022

Abstract

With the increasing availability of large digitized ﬁne art collections,

automated analysis and classiﬁcation of paintings is becoming an inter-

esting area of research. However, due to domain speciﬁcity, implicit sub-

jectivity, and pervasive nuances that vaguely separate art movements,

analyzing art using machine learning techniques poses signiﬁcant chal-

lenges. Residual networks, or variants thereof, are one the most popular

tools for image classiﬁcation tasks, which can extract relevant features for

well-deﬁned classes. In this case study, we focus on the classiﬁcation of a

selected painting ’Portrait of the Painter Charles Bruni’ by Johann Ku-

petzky and the analysis of the performance of the proposed classiﬁer. We

show that the features extracted during residual network training can be

useful for image retrieval within search systems in online art collections.

Keywords: computational creativity; deep learning; feature extraction;

image analysis; machine perception; painting classiﬁcation; residual networks;

transfer learning.

∗Corresponding author: kvak@mail.muni.cz

arXiv:2210.15300v1 [cs.MM] 27 Oct 2022

1 Introduction

Image classiﬁcation is one of the most widely used computer vision tasks. [Lu

and Weng, 2007] In the recent past, deep learning has been very successful

in various visual tasks, such as agent-based simulation of autonomous vehicles

[Schwarting et al., 2018] or computer-aided detection / diagnosis in the health-

care segment. [Doi, 2007] The extensive digitization that has occurred in the

last two decades [Aydoğan, 2019] has led to the question of whether the cura-

tion segment can also be automated using machine methods. The conversion

of information from physical works of art into digital image format plays a key

role in the opening of new research challenges in the interdisciplinary ﬁeld of

computer vision, machine learning, and art history. [Cetinic et al., 2018, Tan

et al., 2016, Saleh and Elgammal, 2015]

Diﬀerent convolutional neural network (CNN) architectures have been proven

to work well for image recognition and classiﬁcation tasks. The basic idea is that

neurons in the visual cortex process images into increasingly complex shapes.

[Lindsay, 2021] The image is ﬁrst segmented at edge boundaries using a light /

dark interface, then merged into simple shapes, and ﬁnally merged into recogniz-

able complex features in subsequent layers. [Albawi et al., 2017] Individual class

labels may be based on some low-level features such as color, texture, or shape,

but are most often based on higher-level features such as semantic description,

activity, or artistic style. [O’Shea and Nash, 2015] CNN tries to mimic this idea

using several layers of artiﬁcial neurons. The standard architecture includes

several convolutional layers that segment the image into small chunks that can

be easily processed. [Albawi et al., 2017]

2 Proposed Method

The use of machine learning for automatic classiﬁcation of ﬁne art collections

has received little attention in the literature so far. [Arora and Elgammal, 2012,

Rodriguez et al., 2018] In recent years, libraries, museums, galleries, and art

centers have been digitizing their collections to promote public interest in the

arts and facilitate access to masterpieces from the comfort of home, a trend that

has been further reinforced by the ongoing COVID-19 pandemic. [Habsary et al.,

2021] These activities create a demand for automated analysis and classiﬁcation

of digitized art. [Khoronko and Mokina, 2021] In this paper, we propose a

novel approach to using CNN output to classify visual artwork. Using CNN

pre-trained on ImageNet,1we consider feature maps computed at the level of

several diﬀerent layers before fully connected layers and compare the perception

of artiﬁcial intelligence with the analysis of art historians and curators. We

show that the extracted features are eﬀective for classifying artists and styles and

1ImageNet is a large-scale visual database designed for use in image classiﬁcation and

object recognition research. The project includes more than 14 million images that have been

manually annotated to indicate what objects are shown. ImageNet features more than 20,000

categories, with a typical category such as "balloon" or "strawberry" consisting of several

hundred images

provide a detailed visualization and discussion of the suitability and eﬀectiveness

of the diﬀerent layers.

2.1 Transfer Learning

In transfer learning, a neural network is ﬁrst trained on a generic dataset (e.g.

ImageNet visual database), and the features learned from the initial task are

transferred to a new network that is ﬁne-tuned for a speciﬁc task. [Weiss et al.,

2016] Deploying pre-trained models on similar data has shown solid results in

image classiﬁcation-related tasks. [Weiss et al., 2016, Zhuang et al., 2020] Sev-

eral organizations have created models such as VGG [Sengupta et al., 2019],

Inception [Szegedy et al., 2016], or ResNet [He et al., 2016] that would take

weeks to train on user-accessible hardware. Pre-trained networks can be down-

loaded and easily ﬁne-tuned to result in lower generalization error while using

less computational eﬀort.

2.2 ResNet50V2 Model Architecture

As deep learning evolves, the structure of neural networks deepens; while this

helps the network to perform more complex feature extraction, it can also in-

troduce the problem of vanishing or exploding gradients. [Joshi et al., 2019]

This can lead to the following drawbacks: (1) Long training time with the con-

vergence of the network becomes very diﬃcult or even non-convergent. (2) The

network performance gradually becomes saturated and even starts to decline.

[Joshi et al., 2019, Kim et al., 2016]

Figure 1: Proposed architecture of ResNet50V2 model.

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

LeveragingComputerVisionApplicationinVisualArts:ACaseStudyontheUseofResidualNeuralNetworktoClassifyandAnalyzeBaroquePaintingsDanielKvak*FacultyofArtsMasarykUniversityBrno,CzechRepublicORCID:0000-0001-7808-7773October28,2022AbstractWiththeincreasingavailabilityoflargedigitizedneartcollections,automa...

展开>> 收起<<

Leveraging Computer Vision Application in Visual Arts A Case Study on the Use of Residual Neural Network to Classify and.pdf

共14页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Leveraging Computer Vision Application in Visual Arts A Case Study on the Use of Residual Neural Network to Classify and

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: