Comparing neural network training performance between Elixir and Python Lucas C. Tavano Lucas K. Aminand Adolfo Gustavo Serra-Seca-Neto

2025-04-29 0 0 874.69KB 18 页 10玖币

侵权投诉

Comparing neural network training performance

between Elixir and Python

Lucas C. Tavano∗

, Lucas K. Amin†and Adolfo Gustavo Serra-Seca-Neto‡

October 2022

Abstract

With a wide range of libraries focused on the machine learning market,

such as TensorFlow, NumPy, Pandas, Keras, and others, Python has made

a name for itself as one of the main programming languages. In February

2021, José Valim and Sean Moriarity published the ﬁrst version of the

Numerical Elixir (Nx) library, a library for tensor operations written in

Elixir. Nx aims to allow the language be a good choice for GPU-intensive

operations. This work aims to compare the results of Python and Elixir on

training convolutional neural networks (CNN) using MNIST and CIFAR-

10 datasets, concluding that Python achieved overall better results, and

that Elixir is already a viable alternative.

1 Introduction

Programming languages have optimized their data structures and libraries for

processing n-dimensional tensors seeking performance and simplicity. Currently,

the market for massively parallel data processing, such as machine learning, is

led by the Python language, followed by R,JavaScript, and C++ [1].

In this scenario, the Elixir language ecosystem has received new tools such

as the Nx library, a new compiler to perform operations in GPU, the ability

to work with ﬂoat16 variables, among other improvements. These new features

enables the Elixir language to participate in the machine learning market. This

work aims to compare the programming languages Elixir and Python in the

context of processing large-volume tensors, comparing execution time, process

parallelization, and machine resources use like Central Processing Unit (CPU),

Random Access Memory (RAM), Graphics Processing Unit (GPU), and Video

RAM (VRAM) in similar algorithms.

∗E-mail: lucas.c.tavano@gmail.com.

†E-mail: lucas_amin_2@hotmail.com.

‡E-mail: adolfo@utfpr.edu.br.

arXiv:2210.13945v1 [cs.LG] 25 Oct 2022

2 Conceptual and Empirical Foundations

This chapter presents theoretical background on tensors, neural networks, datasets,

and the Elixir Nx and the Python NumPy libraries. It also discusses the dif-

ference between functional and object-oriented paradigms in the context of this

project and how this can impact the comparison between Elixir and Python

results.

2.1 Datasets

A dataset is a large amount of data related to the same context [2]. In this

project, we will work with two datasets: MNIST and CIFAR-10, which will be

used to train CNN’s capable of classifying objects in images.

2.1.1 What is the MNIST dataset

MNIST is a labeled set of images of handwritten digits containing 70 thousand

entries; 60 thousand represent the training set, and 10 thousand represent the

test set. Each image in this collection has a resolution of 28x28 pixels; thus, it

is represented in code by a 28 by 28 matrix. Each value contained in this matrix

ranges from 0 to 255, 0 represents black, and 255 is white.

Figure 1: Example of MNIST set images plotted graphically with the Matplotlib

tool

2.1.2 What is the CIFAR-10 dataset

CIFAR-10, as MNIST, represents a large dataset of images; however, in the

case of CIFAR-10, the image set is related to objects and animals. The present

classes in the CIFAR-10 dataset are airplane, automobile, bird, cat, deer, dog,

frog, horse, ship, and truck. It consists of 66,000 images, split 90% for training

and 10% for testing. Each image comprises a 32x32x3 matrix, representing a 32

by 32 RGB image.

Figure 2: Example of CIFAR-10 set images plotted graphically with the Mat-

plotlib tool

2.2 Neural Networks

It can be represented by a multi-layered directional graph, where each node is

referred as a neuron and is responsible for an mathematical operation.

Artiﬁcial neural networks are systems inspired by biological neural networks

constructed for data processing. It can be represented by a multi-layered direc-

tional graph, where each node is referred as a neuron and is responsible for an

mathematical operation [5]. A neural network is designed to be trained on a

speciﬁc dataset, performing changes on its node operations with the objective

of searching the neuron weight set that has the lowest error compared to the

desired labeled result on the dataset [5].

2.2.1 Convolutional Neural Networks

A Convolutional Neural Network is a speciﬁc type of neural network where each

node is responsible for a convolution operation on the received input. An image

vector is used as input and processed through the network into a ﬁnal score

represented on the output layer [5].

2.3 Trained Model Structures

The CNNs used to compare the performance metrics between Python and Elixir

are present in the Axon library examples designed for model creation and train-

ing.

2.3.1 MNIST Model Strucute

This CNN is a straightforward network with two dense layers and a dropout

between them1, as exempliﬁed:

1Axon.input({nil, 784}, "input")

2|> Axon.dense(128, activation: :relu)

3|> Axon.dropout(rate:0.5)

4|> Axon.dense(10, activation: :softmax)

2.3.2 CIFAR-10 Model Strucute

It consists of a CNN on two convolutional layers followed by batch normalization

and max-pooling layers and two dense layers with a dropout between them to

avoid overﬁtting 2, as exempliﬁed:

1Axon.input({nil, 3, 32, 32}, "input")

2|> Axon.conv(32, kernel_size:{3, 3}, activation: :relu)

3|> Axon.batch_norm()

4|> Axon.max_pool(kernel_size:{2, 2})

5|> Axon.conv(64, kernel_size:{3, 3}, activation: :relu)

6|> Axon.batch_norm()

7|> Axon.max_pool(kernel_size:{2, 2})

8|> Axon.flatten()

9|> Axon.dense(64, activation: :relu)

10 |> Axon.dropout(rate:0.5)

11 |> Axon.dense(10, activation: :softmax)

3 Project

This section presents the three sub-projects that we created:

•An Elixir sub-project able to train CNNs able classify MNIST and CIFAR-

10 datasets;

•A Python sub-project able to train CNNs able classify MNIST and CIFAR-

10 datasets;

•An Elixir sub-project able to extract performance benchmarks during the

CNNs train.

Figure 3 is a visual representation of how the sub-projects interact and how

they compose the needed stakeholders for this project. Also, we will explain the

performance benchmark step in Section 4.

1Available at: https://github.com/elixir-nx/axon/blob/main/examples/vision/mnist.exs

2Available at: https://github.com/elixir-nx/axon/blob/main/examples/vision/cifar10.exs

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

ComparingneuralnetworktrainingperformancebetweenElixirandPythonLucasC.Tavano*,LucasK.AminandAdolfoGustavoSerra-Seca-NetoOctober2022AbstractWithawiderangeoflibrariesfocusedonthemachinelearningmarket,suchasTensorFlow,NumPy,Pandas,Keras,andothers,Pythonhasmadeanameforitselfasoneofthemainprogrammingla...

展开>> 收起<<

Comparing neural network training performance between Elixir and Python Lucas C. Tavano Lucas K. Aminand Adolfo Gustavo Serra-Seca-Neto.pdf

共18页,预览4页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Comparing neural network training performance between Elixir and Python Lucas C. Tavano Lucas K. Aminand Adolfo Gustavo Serra-Seca-Neto

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: