Deep Data Augmentation for Weed Recognition Enhancement A Di usion Probabilistic Model and Transfer Learning Based Approach Dong Chena Xinda Qia Yu Zhenga Yuzhen Lub Zhaojian Lic

2025-05-08 0 0 5.9MB 15 页 10玖币

侵权投诉

Deep Data Augmentation for Weed Recognition Enhancement: A Diﬀusion

Probabilistic Model and Transfer Learning Based Approach

Dong Chena, Xinda Qia, Yu Zhenga, Yuzhen Lub, Zhaojian Lic

*Zhaojian Li (lizhaoj1@egr.msu.edu) is the corresponding author

aDepartment of Electrical and Computer Engineering, Michigan State University, East Lansing, MI 48824, USA

bDepartment of Agricultural and Biological Engineering, Mississippi State University, Mississippi State 39762, MS, USA

bDepartment of Biosystems and Agricultural Engineering, Michigan State University, MI 48824, USA

cDepartment of Mechanical Engineering, Michigan State University, East Lansing, MI 48824, USA

Abstract

Weed management plays an important role in many modern agricultural applications. Conventional weed control

methods mainly rely on chemical herbicides or hand weeding, which are often cost-ineﬀective, environmentally un-

friendly, or even posing a threat to food safety and human health. Recently, automated/robotic weeding using machine

vision systems has seen increased research attention with its potential for precise and individualized weed treatment.

However, dedicated, large-scale, and labeled weed image datasets are required to develop robust and eﬀective weed

identiﬁcation systems but they are often diﬃcult and expensive to obtain. To address this issue, data augmentation

approaches, such as generative adversarial networks (GANs), have been explored to generate highly realistic images

for agricultural applications. Yet, despite some progresses, those approaches are often complicated to train or have

diﬃculties to preserve ﬁne details in images. In this paper, we present the ﬁrst work of applying diﬀusion proba-

bilistic models (also known as diﬀusion models) to generate high-quality synthetic weed images based on transfer

learning. Comprehensive experimental results show that the developed approach consistently outperforms several

state-of-the-art GAN models, representing the best trade-oﬀbetween sample ﬁdelity and diversity and the highest

Fr´

echet Inception Distance (FID) score on a common weed dataset, CottonWeedID15 (a dataset speciﬁed to cotton

production systems). In addition, the expanding dataset with synthetic weed images is able to apparently boost model

performance on four deep learning (DL) models for the weed classiﬁcation tasks. Furthermore, the DL models trained

on the CottonWeedID15 dataset with only 10% of real images and 90% of synthetic weed images achieve a testing

accuracy of over 94%, showing high-quality of the generated weed samples. The codes of this study are made publicly

available at https://github.com/DongChen06/DMWeeds.

Keywords: Weed management, weed recognition, diﬀusion probabilistic models, transfer learning, computer vision,

precision agriculture, generative adversarial networks

1. Introduction

Weeds are one of the major threats to crop production.

Weeds compete with crops for resources, resulting in crop

yield and quality losses (Zimdahl, 2007). To increase

crop yields and overcome the shortage of hand weed-

ers, chemical herbicides are being rapidly used to pre-

vent weeds from excessive growth. It is reported that the

global herbicide market grew by 39% between 2002 and

2011 (Rao et al., 2017) and is expected to reach $37.99

billion in 2025 (Ciriminna et al., 2019). However, there

is now overwhelming evidence that intensive applications

of chemicals pose signiﬁcant risks to humans, food safety,

and environment (e.g., surface/underwater, soil, and air

contamination) (Aktar et al., 2009), as well as increas-

ing the evolution of herbicide-resistant weeds, resulting

in signiﬁcant management costs. Therefore, there are

urging needs to develop eﬀective and sustainable weed

management systems to reduce the environmental impact

and selection pressure for weed resistance to herbicides.

Recently, robotic weeding combining machine vision

and traditional mechanical weeding has emerged as a po-

tential solution for site-speciﬁc and individualized weed

management (Fennimore and Cutulle, 2019). Accurate

recognition and localization of weeds is the key to achiev-

ing precise weed control and the development of eﬃ-

Preprint submitted to Journal October 19, 2022

arXiv:2210.09509v1 [cs.CV] 18 Oct 2022

cient and robust robotic weeding systems. To that end,

various deep neural networks (DNNs) have been devel-

oped and attained great successes in various weed detec-

tion and classiﬁcation tasks. Speciﬁcally, the authors in

(Espejo-Garcia et al., 2020a) report that pretraining deep

learning (DL) models on agricultural datasets are advan-

tageous to reduce training epochs and improve model ac-

curacy. Four DL models are ﬁne-tuned on two datasets,

Plant Seedlings Dataset and Early Crop Weeds Dataset

(Espejo-Garcia et al., 2020a), and classiﬁcation perfor-

mance improvements from 0.51% to 1.89% are reported.

In addition, two DL models, ResNet50 (He et al., 2016)

and Inception-v3 (Szegedy et al., 2016), are tested on

the DeepWeeds dataset (Olsen et al., 2019) using trans-

fer learning (Zhang et al., 2022) for weed identiﬁcation,

achieving classiﬁcation accuracies over 95%. In Chen

et al. (2022), 35 state-of-the-art DL models are evaluated

on the CottonWeedID15 dataset for classifying 15 com-

mon weed species in the U.S. cotton production systems,

and 10 out of the 35 models obtain F1 scores over 95%.

Large-scale and diverse labeled image data is essential

for developing the aforementioned DL algorithms. Ef-

fective, robust, and advanced DL algorithms for weed

recognition in complex ﬁeld environment require a com-

prehensive dataset that covers diﬀerent lighting/ﬁeld con-

ditions, various weed shapes/sizes/growth stages/colors,

and mixed camera capture angles (Zhang et al., 2022;

Chen et al., 2022). In contrast, insuﬃcient or low-

quality datasets will lead to poor model generalizability

and overﬁtting issues (Shorten and Khoshgoftaar, 2019).

Recently, several progresses have been made on devel-

oping dedicated image datasets for weed control (Lu

and Young, 2020), such as DeepWeeds (Olsen et al.,

2019), Early Crop Weeds Dataset (Espejo-Garcia et al.,

2020b), early crop dataset(Espejo-Garcia et al., 2020b),

CottonWeedID15 (Chen et al., 2022), and YOLOWeeds

(Dang et al., 2022), just to name a few. However, col-

lecting a large-scale and labeled image dataset is often

resource intensive, time consuming, and economically

costly. One sound approach to addressing this bottleneck

is to develop advanced data augmentation techniques that

can generate high-quality and diverse images (Xu et al.,

2022). However, basic data augmentation approaches,

such as geometric (e.g., ﬂips and rotations), color trans-

formations (e.g., Fancy PCA (Krizhevsky et al., 2012)

and color channel swapping), tend to produce high-

correlated samples and are unable to learn the variations

or invariant features across the samples in the training

data (Shorten and Khoshgoftaar, 2019; Lu and Young,

2020).

Lately, advanced data augmentation approaches, such

as generative adversarial networks (GANs), have gained

increased attention in the agricultural community due

to their capability of generating naturalistic images (Lu

et al., 2022). In Espejo-Garcia et al. (2021), Deep Con-

volutional GAN (DCGAN) (Radford et al., 2015) com-

bined with transfer learning is adopted to generate new

weed images for weed identiﬁcation tasks. The authors

then train the Xception network (Chollet, 2017) with the

synthetic images on the Early Crop Weed dataset (Espejo-

Garcia et al., 2020b) (contains 202 tomato and 130 black

nightshade images at early growth stages) and obtain the

testing accuracy of 99.07%. Conditional Generative Ad-

versarial Network (C-GAN) (Mirza and Osindero, 2014)

is adopted in (Abbas et al., 2021) to generate synthetic

tomato plant leaves to enhance the performance of plant

disease classiﬁcation. The DenseNet121 model (Huang

et al., 2017) is trained on synthetic and real images using

transfer learning on PlantVillage dataset (Hughes et al.,

2015) to classify the tomato leaves images into ten cate-

gories of diseases, yielding 1-4% improvement in classiﬁ-

cation accuracy compared to training on the original data

without image augmentation. The readers are referred

to (Lu et al., 2022) for more recent advances of GANs

in agricultural applications. While impressive progresses

have been made, GANs often suﬀer from training insta-

bility and model collapse issues (Arjovsky et al., 2017;

Creswell et al., 2018; Mescheder et al., 2017), and they

could fail to capture a wide data distribution (Zhao et al.,

2018), which make GANs diﬃcult to be scaled and ap-

plied to new domains.

On the other hand, diﬀusion probabilistic models (also

known as diﬀusion models) have quickly gained popu-

larity in producing high-quality images (Ho et al., 2020;

Song et al., 2020; Dhariwal and Nichol, 2021). Diﬀu-

sion models, inspired by non-equilibrium thermodynam-

ics (Jarzynski, 1997; Sohl-Dickstein et al., 2015), pro-

gressively add noise to data and then construct the de-

sired data samples by a Markov chain from white noise

(Song et al., 2020). Recent researches show that diﬀusion

models are able to produce high-quality synthetic images,

surpassing GANs on several diﬀerent tasks (Dhariwal and

Nichol, 2021), and there is signiﬁcant interest in validat-

ing diﬀusion models in diﬀerent image generation tasks,

such as video generation (Ho et al., 2022b), medical im-

age generation ( ¨

Ozbey et al., 2022), and image-to-image

translation (Saharia et al., 2021). However, the potential

of diﬀusion methods in agricultural image generation re-

mains largely unexplored, partly owing to the substantial

computational burden of image sampling in regular diﬀu-

sion models. In this paper, we presented the ﬁrst results

on image generation for weed recognition tasks using

a classiﬁer-guided diﬀusion model (ADM-G, (Dhariwal

and Nichol, 2021)) based on a 2D U-Net (Ronneberger

et al., 2015) diﬀusion model architecture. A common

weed dataset, CottonWeedID15 (Chen et al., 2022), is

used and evaluated on the image generation and classi-

ﬁcation tasks. The main contributions and the technical

advancements of this paper are highlighted as follows.

1. We present the ﬁrst study on applying diﬀusion

models through transfer learning to generate high-

quality images for weed recognition tasks based on

a popular weed dataset, CottonWeedID15.

2. A post-processing approach based on realism score

is developed and employed to automatically remove

low-quality samples and underrepresented samples

after the training to improve the quality of the gen-

erated samples.

3. Four DL models are evaluated on the expanding

dataset with synthetic images through transfer learn-

ing, showing signiﬁcant performance improvement

on the weed classiﬁcation accuracy.

4. We conduct comprehensive experiments, and the re-

sults show that the proposed approach consistently

outperforms several state-of-the-art GANs in terms

of sample quality and diversity. The codes of this

study are open-sourced at: https://github.com/

DongChen06/DMWeeds.

The rest of the paper is organized as follows. Section

2 presents the dataset and technical details used in this

study. Experimental results and analysis are presented

in Section 3. Finally, concluding remarks and potential

future works are provided in Section 4.

2. Materials and Methods

2.1. CottonWeedID15 dataset

To demonstrate the eﬀectiveness of the proposed ap-

proach, we evaluate the developed approach on a com-

mon weed dataset, CottonWeedID15 (Chen et al., 2022),

which is an open-source weed dataset for weed recogni-

tion tasks collected under natural light conditions at var-

ious weed growth stages in the southern United States,

containing 5,187 images of 15 common weed classes in

the cotton production systems. The dataset has unbal-

anced classes and the images are saved in “jpg” format at

varied resolutions. It includes ﬁve major classes, Morn-

ingglory, Carpetweed, Palmer Amaranth, Waterhemp,

and Purslane, with image numbers ranging from 450 to

1,115, while the minority classes, such as Swinecress and

Spurred Anoda, have less than 100 images. To mitigate

the eﬀect of unbalanced classes and facilitate the model

training, we only included the top 10 weed classes (4,685

images) in this study and each weed class contains at least

200 images. The dataset was randomly partitioned into

training (90%) and testing (10%) sets, in which the train-

ing subset was used to train the diﬀusion models (Sec-

tion 3.1) as well as GAN baselines (Section 3.2) whereas

the testing set was hold out to test the performance of the

DL weed classiﬁers (Section 3.3). To save computation

resources, all weed images were resized to 256×256. The

sample images from CottonWeedID15 dataset are shown

in the top three rows in Figure 2.

2.2. Diﬀusion models

The concept of diﬀusion is widely used in many

ﬁelds, such as physics (Libbrecht, 2005), thermodynam-

ics (Qian, 2015), statistics (Florens-Zmirou, 1989), and

ﬁnance (Eraker, 2001), referred to as a process transfer-

ring from one distribution to another distribution. For ex-

ample, in thermodynamics, diﬀusion process often refers

to the movement of molecules from a region of high

concentration to another region of lower concentration

(Qian, 2015). In Sohl-Dickstein et al. (2015), the authors,

inspired by non-equilibrium statistical physics (Jarzyn-

ski, 1997), formulate the diﬀusion process as a Markov

chain and gradually convert data distribution into another

known distribution (e.g., Gaussian) through an iterative

forward diﬀusion process, then construct new data from

the random noise via a reverse diﬀusion process, the de-

tails of which will be discussed next.

2.2.1. Forward diﬀusion process

Forward diﬀusion process (Song et al., 2020) is the

process that transfers a complex data distribution pdata to

a simple, known distribution pprior, e.g., pprior ∼ N(0,I).

Speciﬁcally, it ﬁrst samples a data point from the real data

distribution x0∼q(x) and gradually adds a small amount

of noise to the sample in Tsteps, producing a sequence

of noisy samples x1,x2,...,xT. The probability of trans-

forming from sample xt−1to xtis modeled by q(xt|xt−1),

and the objective is that when Tapproaches +∞,xTwill

approach the known distribution pprior. In Sohl-Dickstein

et al. (2015), the authors show that q(xt|xt−1) follows a

Gaussian distribution with the following form:

q(xt|xt−1)=N(xt;p1−βtxt−1, βtI),(1)

where βt∈(0,1) is the diﬀusion rate at step tand it can

be held constant or learned online (Ho et al., 2020). Ap-

plying reparameterization tricks (Kingma et al., 2015),

the above process is shown to be the same as xt=

p1−βtxt−1+βtzt−1, i.e., combing xt−1and standard nor-

mal distribution zt−1∼ N(0,I), where βtcontrols the

amount of perturbations. Let at=1−βt, ¯at=QT

i=1ai,

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

DeepDataAugmentationforWeedRecognitionEnhancement:ADiusionProbabilisticModelandTransferLearningBasedApproachDongChena,XindaQia,YuZhenga,YuzhenLub,ZhaojianLic*ZhaojianLi(lizhaoj1@egr.msu.edu)isthecorrespondingauthoraDepartmentofElectricalandComputerEngineering,MichiganStateUniversity,EastLansing,MI4...

展开>> 收起<<

Deep Data Augmentation for Weed Recognition Enhancement A Di usion Probabilistic Model and Transfer Learning Based Approach Dong Chena Xinda Qia Yu Zhenga Yuzhen Lub Zhaojian Lic.pdf

共15页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Deep Data Augmentation for Weed Recognition Enhancement A Di usion Probabilistic Model and Transfer Learning Based Approach Dong Chena Xinda Qia Yu Zhenga Yuzhen Lub Zhaojian Lic

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: