AutoML for Climate Change A Call to Action Renbo Tu1 Nicholas Roberts2 Vishak Prasad3 Sibasis Nayak3 Paarth Jain3 Frederic Sala2 Ganesh Ramakrishnan3 Ameet Talwalkar4 Willie Neiswanger5 Colin White6

2025-04-27 0 0 659.87KB 13 页 10玖币

侵权投诉

AutoML for Climate Change: A Call to Action

Renbo Tu1∗, Nicholas Roberts2, Vishak Prasad3, Sibasis Nayak3, Paarth Jain3,

Frederic Sala2, Ganesh Ramakrishnan3, Ameet Talwalkar4, Willie Neiswanger5, Colin White6

1University of Toronto, 2University of Wisconsin, 3IIT Bombay,

4Carnegie Mellon University, 5Stanford University 6Abacus.AI

Abstract

The challenge that climate change poses to humanity has spurred a rapidly developing

ﬁeld of artiﬁcial intelligence research focused on climate change applications. The climate

change AI (CCAI) community works on a diverse, challenging set of problems which often

involve physics-constrained ML or heterogeneous spatiotemporal data. It would be desirable to

use automated machine learning (AutoML) techniques to automatically ﬁnd high-performing

architectures and hyperparameters for a given dataset. In this work, we benchmark popular

AutoML libraries on three high-leverage CCAI applications: climate modeling, wind power

forecasting, and catalyst discovery. We ﬁnd that out-of-the-box AutoML libraries currently

fail to meaningfully surpass the performance of human-designed CCAI models. However, we

also identify a few key weaknesses, which stem from the fact that most AutoML techniques are

tailored to computer vision and NLP applications. For example, while dozens of search spaces

have been designed for image and language data, none have been designed for spatiotemporal

data. Addressing these key weaknesses can lead to the discovery of novel architectures that yield

substantial performance gains across numerous CCAI applications. Therefore, we present a call

to action to the AutoML community, since there are a number of concrete, promising directions

for future work in the space of AutoML for CCAI. We release our code and a list of resources at

https://github.com/climate-change-automl/climate-change-automl.

1 Introduction

There is an increasing body of evidence which shows that climate change is one of the biggest

threats facing humanity today [

]. Taking action towards climate change must come in

many forms, such as reducing greenhouse gases and facilitating the adaption of renewable energy. A

rapidly developing area of artiﬁcial intelligence research, climate change AI (CCAI), is focused on

applications to mitigate the eﬀects of climate change [11,26,38].

On the other hand, the automated machine learning (AutoML) community has been focused

on designing eﬃcient algorithms for problems such as hyperparameter optimization (HPO) and

neural architecture search (NAS) [

]. In general, the goal of AutoML is to develop algorithms that

automate the process of designing architectures and tuning hyperparameters for a given dataset.

Although AutoML would seemingly be most useful on understudied datasets where there is less

human intuition [36,47], most AutoML techniques, whether implicitly or explicitly, are tailored to

CV and NLP tasks. Furthermore, a few recent works show that state-of-the-art AutoML techniques

∗

Work done while ﬁrst author was part-time at Abacus.AI. Correspondence to: Colin White <

colin@abacus.ai

arXiv:2210.03324v1 [cs.LG] 7 Oct 2022

AutoML Methods CCAI Benchmarks Metrics of Interest

ClimART (CA)

Open Catalyst Project

(OC20)

Wind Power Forecasting

(SDWPF)

Accuracy,

Inference Latency

Mean Absolute Error

Between Energies

Average Accuracy

Across Turbines

Hyperparameter

Optimization (Optuna)

Neural Architecture

Search (SMAC3)

Figure 1: Overview of the main components of our study.

for common CV-based tasks do not transfer to other non-CV tasks [

]. A natural question is

therefore, are AutoML techniques beneﬁcial for high-impact CCAI applications?

In this work, we benchmark popular AutoML libraries on three high-leverage CCAI tasks: climate

modeling, wind power forecasting, and catalyst discovery (see Fig. 1).

Across several experiments, we are unable to show that out-of-the-box AutoML techniques

meaningfully surpass the performance of human-designed models. At the same time, we identify

concrete weaknesses stemming from the fact that AutoML techniques have not been designed

for common CCAI themes such as spatiotemporal data or physics-constrained ML. For example,

designing a search space which interpolates among MLPs, CNNs, GNNs, and GCNs (all of which

have been used for climate modeling [

]) would allow NAS algorithms to discover novel

combinations of existing architecture components, potentially leading to substantial performance

gains across several spatiotemporal forecasting applications. Therefore, we give a call to action

to the AutoML community, with the aim of leveraging the full power of AutoML on challenging,

high-impact CCAI tasks.

Related work.

In recent years, several techniques have been developed for atmospheric radiative

transfer [

], wind power forecasting [

], catalyst prediction [

], and many more

areas [24,25,50]. For a survey of machine learning tasks in the climate change space, see [38].

HPO [

] and NAS [

] are two popular areas of AutoML [

]. Recently, Tu et al. introduced

NAS-Bench-360 [

], a benchmark suite to evaluate NAS methods on a diverse set of understudied

tasks, in order to help move the ﬁeld of NAS away from its emphasis on CV and NLP. They showed

that current state-of-the-art NAS methods do not perform well on diverse tasks. Another recent

work similarly showed that the best techniques and hyperparameters on CV-based tasks do not

transfer to more diverse tasks [

]. However, for both of these works, the analyses used a few ﬁxed

search spaces rather than identifying models hand-designed speciﬁcally for each task.

2 Methodology

In this section, we describe our methodology, driven by the following two research questions:

•RQ 1:

Can current out-of-the-box AutoML techniques substantially improve performance

compared to human-designed models in high-leverage climate change AI applications?

•RQ 2: If not, then what are the key limitations and weaknesses of existing techniques?

In order to answer

RQ 1

, we select datasets which (1) correspond to impactful directions in

climate change research, and (2) have existing strong human-designed baselines. For example,

we choose datasets which were recently featured in large competitions, with top solutions now

open-source. We describe the details of each dataset in Section 3.

For each of the datasets we choose, we ﬁrst ﬁnd open-source high-performing human-designed

models. Then we run Optuna [

] or SMAC3 [

], two of the most widely-used AutoML libraries

today, using top human-designed models as the base. We compare the resulting searched models to

top human-designed models.

In order to answer

RQ 2

, we check for general weaknesses in AutoML techniques applied to

CCAI tasks, which can be overcome with future work. For example, we look at whether the AutoML

techniques are limited due to being implicitly tailored to CV tasks.

3 Experiments and Discussion

In this section, for three CCAI tasks, we give a brief description of the task, dataset, and our AutoML

experiments. Then, in Section 3.2, we use our experiments to answer RQ 1 and RQ 2.

3.1 Experimental Setup

Atmospheric Radiative Transfer.

Numerical weather prediction models, as well as global and

regional climate models, give crucial information to policymakers and the public about the impact of

changes in the Earth’s climate. The bottleneck is atmospheric radiative transfer (ART) calculations,

which are used to compute the heating rate of any given layer of the atmosphere. While ART has

historically been calculated using computationally intensive physics simulations, researchers have

recently used neural networks to substantially reduce the computational bottleneck, enabling ART

to be run at ﬁner resolutions and obtaining better overall predictions.

We use the ClimART dataset [

] from the NeurIPS Datasets and Benchmarks Track 2021. It

consists of global snapshots of the atmosphere across a discretization of latitude, longitude, atmo-

spheric height, and time from 1979 to 2014. Each datapoint contains measurements of temperature,

water vapor, and aerosols. Prior work has tested MLPs, CNNs, GNNs, and GCNs as baselines [6].

We run HPO on the CNN baseline from Cachay et al. [

] using the Optuna library [

]. The CNN

model is chosen because it had the lowest RMSE and second-lowest latency out of all ﬁve baselines

from Cachay et al. We tune learning rate, weight decay, dropout, and batch size. We also run NAS

using SMAC3 [

]. We set a categorical hyperparameter to choose among MLP, CNN, GNN, GCN,

and L-GCN [

] while also tuning learning rate and batch size. See Appendix A.1 for more details of

the dataset and experiments.

Wind Power Forecasting.

Wind power is one of the leading renewable energy types, since it is

cheap, eﬃcient, and harmless to the environment [

]. The only major downside in wind power

is its unreliablility: changes in wind speed and direction make the energy gained from wind power

inconsistent. In order to keep the balance of energy generation and consumption on the power grid,

other sources of energy must be added on short notice when wind power is down, which is not always

possible (for example, coal plants take at least 6 hours to start up) [

]. Therefore, forecasting wind

power is an important problem that must be solved to facilitate greater adoption of wind power.

We use the SDWPF (Spatial Dynamic Wind Power Forecasting) dataset, which was recently

featured in a KDD Cup 2022 competition that included 2490 participants [

]. This is by far the

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

AutoMLforClimateChange:ACalltoActionRenboTu1*,NicholasRoberts2,VishakPrasad3,SibasisNayak3,PaarthJain3,FredericSala2,GaneshRamakrishnan3,AmeetTalwalkar4,WillieNeiswanger5,ColinWhite61UniversityofToronto,2UniversityofWisconsin,3IITBombay,4CarnegieMellonUniversity,5StanfordUniversity6Abacus.AIAbstract...

展开>> 收起<<

AutoML for Climate Change A Call to Action Renbo Tu1 Nicholas Roberts2 Vishak Prasad3 Sibasis Nayak3 Paarth Jain3 Frederic Sala2 Ganesh Ramakrishnan3 Ameet Talwalkar4 Willie Neiswanger5 Colin White6.pdf

共13页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

AutoML for Climate Change A Call to Action Renbo Tu1 Nicholas Roberts2 Vishak Prasad3 Sibasis Nayak3 Paarth Jain3 Frederic Sala2 Ganesh Ramakrishnan3 Ameet Talwalkar4 Willie Neiswanger5 Colin White6

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: