INTEGRATING PRE-PROCESSING PIPELINES IN ODC BASED FRAMEWORK U.Otamendi I.Azpiroz M.Quartulli I.Olaizola Vicomtech Foundation

2025-05-05 0 0 2.38MB 4 页 10玖币

侵权投诉

INTEGRATING PRE-PROCESSING PIPELINES IN ODC BASED FRAMEWORK

U.Otamendi, I.Azpiroz, M.Quartulli, I.Olaizola

Vicomtech Foundation

Basque Research and Technology Alliance (BRTA)

Donostia-San Sebasti´

an, 20009, Spain

ABSTRACT

Using on-demand processing pipelines to generate vir-

tual geospatial products is beneﬁcial to optimizing resource

management and decreasing processing requirements and

data storage space. Additionally, pre-processed products im-

prove data quality for data-driven analytical algorithms, such

as machine learning or deep learning models. This paper

proposes a method to integrate virtual products based on inte-

grating open-source processing pipelines. In order to validate

and evaluate the functioning of this approach, we have inte-

grated it into a geo-imagery management framework based

on Open Data Cube (ODC). To validate the methodology,

we have performed three experiments developing on-demand

processing pipelines using multi-sensor remote sensing data,

for instance, Sentinel-1 and Sentinel-2. These pipelines are

integrated using open-source processing frameworks.

Index Terms—SNAP, Sentinel, data exploitation, man-

agement, optimization, machine learning, data processing

1. INTRODUCTION

Geospatial imagery is widely used in multiple ﬁelds of en-

vironmental management approaches based on modern com-

puting, such as deep learning [1]. For instance, periodic data

provided by satellites are useful for analysis and pattern ex-

traction from a time series. The method offers a more accurate

understanding of the evolution of the explored area. How-

ever, these high spatial resolution data require a large storage

capacity. In addition, the processing of these data is compu-

tationally demanding [2, 3].

Productive geo-imagery processing for rapid mapping is

highly dependent on the efﬁciency of local statistics gener-

ation from remote sensing images. An automated computa-

tion supposes a substantial advance for agronomists, scien-

tists, and satellite-derived data users.

In a previous paper [4] we proposed a methodology to

address the limitations of non-expert users in managing and

processing remote sensing and geo-imagery data. This sys-

tem automatically ingests geospatial data and allows non-

expert users to manage geospatial data in data-driven algo-

rithms without requiring knowledge of remote sensing or

Fig. 1: The ﬁgure shows an overview of the proposed method-

ology to generate on-demand geospatial virtual products via

processing pipelines. As shown, the non-expert user can de-

clare a virtual product. Then, the framework uses the avail-

able processing operations to create a processing pipeline that

converts the source geospatial data to the desired format. Fi-

nally, the resulting product is ingested by the Open Data Cube

architecture, allowing the non-expert user to use the data in

analytical processes.

geo-imagery exploitation. However, this considerably limits

the exploration capability of modiﬁed products. Conse-

quently, a non-expert user will only be limited to analyzing

those products that the satellite imagery distributors have

previously deﬁned.

Therefore, this hinders the optimal use of the data in the

performance of the algorithmic processes. In this sense, the

main goal of the current contribution is to describe the inte-

gration of on-demand processing pipelines in an ODC-based

infrastructure (see ﬁg. 1). This approach provides several

beneﬁts of resource optimization and data quality improve-

ment. Additionally, users acquire the ability to create virtual

geospatial data based on processing pipelines to automatically

generate adequate data to train and use data-driven models

[5].

The implementation of this methodology has been inte-

grated with the Open Data Cube (ODC) based architecture

proposed in the previous data management paper [4]. In or-

der to validate this approach, we have performed three experi-

arXiv:2210.01528v1 [cs.CV] 4 Oct 2022

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

INTEGRATINGPRE-PROCESSINGPIPELINESINODCBASEDFRAMEWORKU.Otamendi,I.Azpiroz,M.Quartulli,I.OlaizolaVicomtechFoundationBasqueResearchandTechnologyAlliance(BRTA)Donostia-SanSebasti´an,20009,SpainABSTRACTUsingon-demandprocessingpipelinestogeneratevir-tualgeospatialproductsisbenecialtooptimizingresourcema...

展开>> 收起<<

INTEGRATING PRE-PROCESSING PIPELINES IN ODC BASED FRAMEWORK U.Otamendi I.Azpiroz M.Quartulli I.Olaizola Vicomtech Foundation.pdf

共4页,预览1页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

INTEGRATING PRE-PROCESSING PIPELINES IN ODC BASED FRAMEWORK U.Otamendi I.Azpiroz M.Quartulli I.Olaizola Vicomtech Foundation

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: