iquaflow A new framework to measure image quality

2025-05-03 0 0 3.01MB 19 页 10玖币

侵权投诉

IQUAFLOW: A NEW FRAMEWORK TO MEASURE IMAGE QUALITY

A PREPRINT

P. Gallés1, K. Takáts1, M. Hernández-Cabronero2, D. Berga3, L. Pega1, L. Riordan-Chen1, C. Garcia1, G. Becker1, A.

Garriga3, A. Bukva3, J. Serra-Sagristà2, D. Vilaseca1, and J. Marín1

1Satellogic Inc; pau.galles@satellogic.com

2Universitat Autònoma de Barcelona - UAB-DEIC-GICI; miguel.hernandez@uab.cat

3EURECAT - Multimedia Technologies Unit; david.berga@eurecat.org

October 25, 2022

ABSTRACT

IQUAFLOW is a new image quality framework that provides a set of tools to assess image quality.

The user can add custom metrics that can be easily integrated. Furthermore, IQUAFLOW allows to

measure quality by using the performance of AI models trained on the images as a proxy. This also

helps to easily make studies of performance degradation of several modiﬁcations of the original

dataset, for instance, with images reconstructed after different levels of lossy compression; satellite

images would be a use case example, since they are commonly compressed before downloading to

the ground. In this situation, the optimization problem consists in ﬁnding the smallest images that

provide yet sufﬁcient quality to meet the required performance of the deep learning algorithms. Thus,

a study with IQUAFLOW is suitable for such case. All this development is wrapped in MLFLOW:

an interactive tool used to visualize and summarize the results. This document describes different

use cases and provides links to their respective repositories. To ease the creation of new studies, we

include a cookiecutter

repository. The source code, issue tracker and aforementioned repositories

are all hosted on GitHub 2.

Keywords image quality ·vision ·deep learning ·augmentation ·compression

1 Introduction

The increasing interest and investment in low-cost Earth Observation (EO) satellites (such as nanosats and microsats) in

recent years has made possible the creation and improvement of multiple applications that feed on the images obtained

Buchen [2014]. Thanks to the emergence of new sensors, the quality of these images has increased signiﬁcantly, thus

contributing to the accuracy and efﬁciency of the applications that use them, giving rise to the NewSpace era, which has

led to the emergence of many new companies.

It is the case of Satellogic, a company that was founded in 2010 and specializes in Earth observation data and analytical

imagery solutions. Satellogic designs, builds and operates its own ﬂeet of Earth observation satellites to frequently

collect affordable high-resolution imagery for decision-making in a broad range of industrial, environmental and

government applications. The Satellogic satellite constellation consists of individual small satellites, named NewSats.

Each of the NewSat satellites has a multispectral and a hyperspectral sensor. Its data is used in some of the studies

mentioned in the present article.

Imagery is a means to an end. Large-scale data analytics and artiﬁcial intelligence equipment turns imagery into answers

to help industries, governments and individuals solve problems, facilitate decision making and generate competitive

advantage. In this context, the growth in the number of EO users has also increased the land area of interest and thus

1https://github.com/satellogic/iquaﬂow-use-case-cookiecutter

2https://github.com/satellogic/iquaﬂow

arXiv:2210.13269v1 [cs.CV] 24 Oct 2022

arXiv Template A PREPRINT

the amount of imagery required to meet their needs. As a result, an increasing volume of EO image data needs to be

transmitted. However, the transmission capacity between satellites and ground stations has not grown at the same rate

as the required volume of imagery.

Due to their orbit, satellites can only contact these stations for limited periods of time and with limited bandwidths.

Moreover, the total amount of energy available for all satellite tasks - including image capture, processing and

transmission - is also limited. These limitations pose a bottleneck in the operation of EO satellites since the quantity

and quality of images reaching the ground is determined by the efﬁciency of transmission. In turn, this limitation has a

negative impact both on the costs of EO services and on users and applications that increasingly demand higher quality

and frequency of observation of the regions of interest.

Given the aforementioned needs, this project seeks to optimize the decision making process when selecting an image

processing algorithm to optimize the storage, quality and transmission of images either on EO satellites or on the ground.

Optimization refers to ﬁnding and detecting the satellite image parameters that allow the smallest compressed data

volume that provides sufﬁcient quality to meet the required performance of the deep learning algorithms used. In order

to achieve this objective a new framework named IQUAFLOW (acronym of Image Quality Assessment) is proposed. The

framework includes the necessary tools to draw conclusions based on speciﬁc metrics.

IQUAFLOW consists of multiple Python modules that can be imported in any image quality-related project for research

or production. The framework also includes the necessary tools to automatize and organize experiments. Most of similar

tools help to implement better tractability model trainings in order to perform model selection or model performance

degradation studies (See Lofqvist and Cano [2021], Jo et al. [2021] and Delac et al. [2005]). Instead, IQUAFLOW,

analyzes the change in performance of the models based on modiﬁcations in quality of the training images. Having this

perspective in mind, IQUAFLOW organizes training executions based on qualities of the images. Thus, this approach is

considered image quality selection rather than the common practice of image selection. Additionally, this framework

provides a variety of tools that help to speed up the machine learning development such as dataset sanity check, dataset

statistics, dataset visualization and dataset modiﬁcation algorithms. It is designed to easily adapt to conventions. The

tool can be imported into any kind of deep learning framework such as tensorﬂow, keras or pytorch. This allows to

effortlessly generate experiments on any existing project regardless of its dependencies. Internally, this project relies

in MLFLOW, which enables to organize the information locally or even in a remote server with an interaction that

is abstracted to the user. As a result, the user does not need to learn another cumbersome framework. Ultimately,

IQUAFLOW only needs logging parameters that are easy to adapt (such as generating a json or saving ﬁles in a speciﬁc

folder that is indicated as an input argument in the user script). Figure 1 shows the typical workﬂow of a study made

with IQUAFLOW.

Figure 1: Typical use case workﬂow using IQUAFLOW. The ﬁrst step is to overview and sort the data. Then some

modiﬁers can be deﬁned. Afterwards, the task and the overall experiment is set and executed. Then additional metrics

can be estimated on the results. Finally, one can summarize, sort and visualize the results.

2 Use cases

IQUAFLOW has already been used in various use cases to solve real problems that are faced in the EO industry. In this

section we summarize some of its use cases highlighting the beneﬁts of using IQUAFLOW in them. The ﬁrst section

arXiv Template A PREPRINT

(2.1) is actually an example of IQUAFLOW usage rather than a real case. Then the other two sections (2.2 and 2.3) are

describing two groups of use cases related with object detection and super resolution respectively.

2.1 MNIST showcase

The objective of this study

is to examine how the training performance of a deep learning classiﬁer degrades as the

amount of noise in the input dataset is increased. The dataset used is MNIST Deng [2012], which is widely used for

benchmaking deep learning algorithms. It consists of a dataset of handwritten digits (

) images that is used to evaluate

machine learning pattern recognition algorithms. Each digit image is 28 ×28 pixels.

The ﬁrst step in the development process is to prepare the user training script that contains the deep learning classiﬁer.

In this case the model is built using the architecture of a resnet18 He et al. [2015]. Afterwards the user script is adapted

to follow the IQUAFLOW conventions. This means to add the appropriate input arguments (See section 3). The output

that generates the user script also had to follow the standards of IQUAFLOW. Next step is to create a custom modiﬁer

that integrates with IQUAFLOW and does the desired alteration on the original dataset. In this case the modiﬁer is adding

noise following a gaussian distribution. The standard deviation of that noise distribution is an adjustable parameter.

Finally IQUAFLOW executed all requested combinations: hyperparameter variations, number of repetitions and dataset

modiﬁers.

Despite being a classiﬁcation task the loss function was set as Mean Squared Error with the aim to penalize further the

predicted digits that are more distant from the actual labeled numbers. The Figure 2 shows how the performance in the

validation set degrades as the noise amount is increased up to a point where it does not degrade further (around sigma

noise modiﬁer of 2).

Figure 2: Mean Squared Error of the predicted digital label number with respect to the actual target number for different

noise level dataset modiﬁcations. The noise level is deﬁned as sigma of the Gaussian random noise distribution applied

on them.

2.2 Object Detection on compressed satellite images

As mentioned in the introduction EO satellites capturing images have limited energy resources. Further to that they

have limited connection time and data ﬂow capacity between the orbiting equipment and ground station systems on

earth. The images are therefore compressed before downloading. There are compression algorithms that can be reverted

recovering the original uncompressed image (lossless algorithms). On the other hand there are lossy algorithms that will

compress further with irreversible operations (such as interpolating the image to a smaller size with less pixels ). When

3https://github.com/satellogic/iquaﬂow-mnist-use-case

arXiv Template A PREPRINT

the usage of these images is well known the compression can be adjusted. This way, we avoid loss in performance on

its intended application. This can be the case of detecting objects with a deep learning algorithm on satellite images.

We have built three use cases to study how the the application of different compression techniques affects object

detection performance. In the ﬁrst use case

, we sought to replicate the study from Lofqvist and Cano [2021]

where they study the performance of CNN-based object detectors on constrained devices by applying different image

compression techniques to satellite data. In this case they focus on execution times, memory consumption and some

insights about accuracy. In our case we have focused on the performance metrics as a function of compression ratios.

For this sudy the public dataset DOTA Xia et al. [2018] was used. The images were collected from different sensors

with image sizes ranging from

800 ×800

20000 ×20000

pixels while the pixel size varies from

0.3

m to

resolution. DOTA has different versions, in the present study DOTA-v1.0 has been used which contains 15 common

categories,

2806

images and more than

188

k object instances. The proportions of the training, validation, and testing

sets in DOTA-v1.0 are

1/2

1/6

, and

1/3

Xia et al. [2018]. A disadvantage of this dataset is that the test set is not

openly available, rather it is in a form of a remote service to query the predictions. This does not allow to compress the

test images the same way the other partitions are modiﬁed in the present study. To overcome this problem we divided

up the validation set: half of it is used as actual validation and the other half for testing. Then the images are cropped to

1024 ×1024

with padding when necessary. After this operation the amount of crops for the partitions train, validation

and testing are respectively 9734,2670 and 2627.

Our second use case

is similar to the ﬁrst one with newer implementations of object detection algorithms that are

based on oriented annotations. The dataset used was the same as in the ﬁrst study because the original annotations are

oriented.

The ﬁnal objective was to ﬁnd an optimal compression ratio which is deﬁned as the minimum average ﬁle size that

can be set without lowering the performance. We have found this to be around JPEG quality score of 70 (parameter

CV_IMWRITE_JPEG_QUALITY deﬁned in Bradski [2000]) for both horizontal and oriented models. However, the

oriented models had better performances.

Finally a use case is done with a prototype of airplane detection

. In this use case, we apply different compression

algorithms on a new dataset. The objective of this experiment is to determine how much each of the compression

algorithms affects the performance of an airplane detector. The airplanes dataset consists of Satellogic images capturing

airport areas each of

1024 ×1024

pixels. These captures were made using NewSat Satellogic constellation (

m GSD).

The annotations were made using Happyrobot7platform.

Figure 3: JPEG compression effects (original, JPG10,and JPG5 from left to right).

Figure 4: Quantization effects (original, 5 and 2-bits from left to right).

4https://github.com/satellogic/iquaﬂow-dota-use-case

5https://github.com/satellogic/iquaﬂow-dota-obb-use-case

6https://github.com/satellogic/iquaﬂow-airport-use-case

7https://happyrobot.ai

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

IQUAFLOW:ANEWFRAMEWORKTOMEASUREIMAGEQUALITYAPREPRINTP.Gallés1,K.Takáts1,M.Hernández-Cabronero2,D.Berga3,L.Pega1,L.Riordan-Chen1,C.Garcia1,G.Becker1,A.Garriga3,A.Bukva3,J.Serra-Sagristà2,D.Vilaseca1,andJ.Marín11SatellogicInc;pau.galles@satellogic.com2UniversitatAutònomadeBarcelona-UAB-DEIC-GICI;migue...

展开>> 收起<<

iquaflow A new framework to measure image quality.pdf

共19页,预览4页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

iquaflow A new framework to measure image quality

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: