Ten Years after ImageNet A 360Perspective on AI Sanjay Chawla1Preslav Nakov2Ahmed Ali1Wendy Hall3

2025-05-02 0 0 1.85MB 15 页 10玖币

侵权投诉

Ten Years after ImageNet:

A 360◦Perspective on AI

Sanjay Chawla1Preslav Nakov2Ahmed Ali1Wendy Hall3

Issa Khalil1Xiaosong Ma1Husrev Taha Sencar1

Ingmar Weber5Michael Woolridge4Ting Yu1

1Qatar Computing Research Institute

2Mohamed Bin Zayed University of AI

3University of Southampton

4Oxford University

5Saarland University

October 6, 2022

Abstract

It is ten years since neural networks made their spectacular comeback. Prompted by this anniversary, we

take a holistic perspective on Artiﬁcial Intelligence (AI). Supervised Learning for cognitive tasks is eﬀectively

solved — provided we have enough high-quality labeled data. However, deep neural network models are not

easily interpretable, and thus the debate between blackbox and whitebox modeling has come to the fore.

The rise of attention networks, self-supervised learning, generative modeling, and graph neural networks has

widened the application space of AI. Deep Learning has also propelled the return of reinforcement learning

as a core building block of autonomous decision making systems. The possible harms made possible by new

AI technologies have raised socio-technical issues such as transparency, fairness, and accountability. The

dominance of AI by Big-Tech who control talent, computing resources, and most importantly, data may

lead to an extreme AI divide. Failure to meet high expectations in high proﬁle, and much heralded ﬂagship

projects like self-driving vehicles could trigger another AI winter.

arXiv:2210.01797v1 [cs.LG] 1 Oct 2022

Contents

1 Introduction 3

2 Digression: What is AI? 3

3 Is Supervised Learning Solved? 3

3.1 SuccessStories .............................................. 4

3.2 DoubleDescentPhenomenon ...................................... 5

4 Cognitive Content Generation 5

4.1 GeneratingSyntheticImages ...................................... 6

4.2 GeneratingNaturalLanguage...................................... 7

5 Autonomous Decision-Making 7

6 AI Computation: Software and Hardware 9

7 Deep Learning (In)Security 10

7.1 AdversarialAttack ............................................ 10

7.2 Poisoning and Inference Attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

8 AI Socio-Technical Ecosystem 11

8.1 (Un)InterpretableAI........................................... 11

8.2 Sentient AI or Stochastic Parrot? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

8.3 Causality ................................................. 12

8.4 OwnershipofAI ............................................. 12

8.5 Equitability................................................ 12

8.6 NoData,NoAI.............................................. 12

8.7 AIandEducation............................................. 13

9 AI Winter: Back to the Future 13

Figure 1: The original AlexNet architecture used for the ImageNet Challenge in 2012 [2]. The network had

eight layers and sixty million parameters and took six days to train on two GPUs.

1 Introduction

The ImageNet challenge for automatically recognizing and labeling objects in images was launched in 2010 [1].

However, it was in 2012 when AlexNet, an eight-layer (hence deep) convolutional neural network (CNN) emerged

as the winner by a large margin, and ushered in the new era of AI [2]. CNNs were not new and had been proposed

as far back as the 1990s, but had been sidelined in favor of more theoretically rigorous ML approaches such as

support vector machines (SVMs) and boosting methods [3, 4, 5]. So, why did CNNs outperform other models?

Two reasons are usually given. First was the provision of substantial high-quality training data. The ImageNet

database was a one-of-a-kind benchmark and consisted of over fourteen million hand-annotated images from

more than twenty thousand diverse categories. The multilayer CNN had the capacity to eﬀectively memorize

the training subset of ImageNet and, at the same time, generalize to unseen examples — a characteristic that is

not fully understood even today [6]. Second, Graphics Processing Units (GPUs), which were originally designed

for parallelizing image processing tasks, proved to be ideally suited for the computational problems associated

with training CNNs, making it practicable to train deep CNNs on large data sets in a reasonable amount of

time. The combination of Big Data, Big Models, and relatively cheap parallel computation became the mantra

that swept through AI research, in disciplines spanning from astronomy to zoology, and all applications that

have elements of data and prediction.

Our perspective has two parts.

We begin with a high-level, partly technical, overview of the current state of AI. We will begin by reviewing

supervised learning, a machine learning task that has been most impacted by deep learning (DL). We follow

with a discussion on deep content generation models, on the resurrection of reinforcement learning, on the

emergence of specialized software libraries for deep learning, and on the role of GPUs. We will conclude the

ﬁrst part by highlighting how adversarial samples can be designed to fool deep models and whether it is possible

to make models robust.

In part two of the perspective, we consider the many socio-technical issues surrounding AI. Of particular

interest is the dominance of Big Tech on AI. Eﬀectively, only big corporations have the resources (expertise,

computation, and data) to scale AI to a level where it can be meaningfully and accurately applied.

2 Digression: What is AI?

The term Artiﬁcial Intelligence was ﬁrst introduced in 1956 in a workshop proposal submitted by John McCarthy

to the Rockefeller foundation, which proposed that “every aspect of learning or any other feature of intelligence

can in principle be so precisely described that a machine can be made to simulate it [7].” Before that, Alan

Turing in 1947, in an unpublished report titled “Intelligent Machinery”, speculated that “What we want is a

machine that can learn from experience” and suggested that the “possibility of letting the machine alter its own

instructions provides the mechanism for this1.” Much of the recent success in AI is under the distinct subﬁeld

of AI known as Machine Learning and since the role of data is central, there is a broader term, Data Science,

that is often used to subsume related disciplines including Statistics.

3 Is Supervised Learning Solved?

Supervised Learning (SL) is the poster child of success of machine learning. Depending upon the context, SL is

known as classiﬁcation, regression, or prediction. Since the modern advent of deep learning, both the accuracy

1https://www.britannica.com/technology/artiﬁcial-intelligence/Alan-Turing-and-the-beginning-of-AI

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

TenYearsafterImageNet:A360PerspectiveonAISanjayChawla1PreslavNakov2AhmedAli1WendyHall3IssaKhalil1XiaosongMa1HusrevTahaSencar1IngmarWeber5MichaelWoolridge4TingYu11QatarComputingResearchInstitute2MohamedBinZayedUniversityofAI3UniversityofSouthampton4OxfordUniversity5SaarlandUniversityOctober6,2022Abs...

展开>> 收起<<

Ten Years after ImageNet A 360Perspective on AI Sanjay Chawla1Preslav Nakov2Ahmed Ali1Wendy Hall3.pdf

共15页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Ten Years after ImageNet A 360Perspective on AI Sanjay Chawla1Preslav Nakov2Ahmed Ali1Wendy Hall3

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: