Phenomenological Model of Superconducting Optoelectronic Loop Neurons Jerey M. Shainline Bryce A. Primavera and Saeed Khan

2025-05-02 0 0 9.56MB 34 页 10玖币

侵权投诉

Phenomenological Model of

Superconducting Optoelectronic Loop Neurons

Jeﬀrey M. Shainline∗, Bryce A. Primavera, and Saeed Khan

National Institute of Standards and Technology

325 Broadway, Boulder, CO, USA, 80305

∗jeﬀrey.shainline@nist.gov

October 17, 2022

Abstract

Superconducting optoelectronic loop neurons are a class of circuits potentially conducive to networks for large-scale

artiﬁcial cognition. These circuits employ superconducting components including single-photon detectors, Josephson

junctions, and transformers to achieve neuromorphic functions. To date, all simulations of loop neurons have used

ﬁrst-principles circuit analysis to model the behavior of synapses, dendrites, and neurons. These circuit models are

computationally ineﬃcient and leave opaque the relationship between loop neurons and other complex systems. Here

we introduce a modeling framework that captures the behavior of the relevant synaptic, dendritic, and neuronal circuits

at a phenomenological level without resorting to full circuit equations. Within this compact model, each dendrite

is discovered to obey a single nonlinear leaky-integrator ordinary diﬀerential equation, while a neuron is modeled as

a dendrite with a thresholding element and an additional feedback mechanism for establishing a refractory period.

A synapse is modeled as a single-photon detector coupled to a dendrite, where the response of the single-photon

detector follows a closed-form expression. We quantify the accuracy of the phenomenological model relative to circuit

simulations and ﬁnd that the approach reduces computational time by a factor of ten thousand while maintaining

accuracy of one part in ten thousand. We demonstrate the use of the model with several basic examples. The net

increase in computational eﬃciency enables future simulation of large networks, while the formulation provides a

connection to a large body of work in applied mathematics, computational neuroscience, and physical systems such

as spin glasses.

1 Introduction

A technological platform capable of realizing networks at

the scale and complexity of the brains of intelligent organ-

isms would be a tool of supreme scientiﬁc utility. Neu-

romorphic hardware based on conventional silicon micro-

electronics has a great deal to oﬀer in this regard [1–5].

Yet challenges remain, primarily concerning bottlenecks

in the shared communication infrastructure that must be

employed to emulate the connectivity of biological neu-

rons. Alternative hardware may bring new beneﬁts, and

we have argued elsewhere for the advantages of a super-

conducting optoelectronic approach [6–11]. In brief, light

for communication enables high fan-out with low latency

across spatial scales from a chip to a many-wafer system.

Superconducting electronics provide single-photon detec-

tion coupled to high-speed, low-energy analog neuromor-

phic computational primitives.

While the components of these superconducting op-

toelectronic networks (SOENs) have been demonstrated

[12–15], full neurons have not. Prior to undertaking the

eﬀort and expense of realizing the requisite semiconductor-

superconductor-photonic fabrication process, it is prudent

to gain conﬁdence that SOENs are indeed ripe for further

investigation. This conﬁdence can be gained through sim-

ulations of device, circuit, and system behavior using nu-

merical simulations on digital computers. The constituent

devices are most commonly modeled with circuit simula-

tions carried out on a picosecond time scale to accurately

capture the dynamics of Josephson junctions. Simulation

of networks of large numbers of these neurons becomes

computationally intensive. From the perspective of the

neural system, the picosecond dynamics of the JJs are not

of primary interest, and one would prefer to treat each

synapse, dendrite, and neuron as an input-output device

with a model that accurately captures the circuit dynam-

ics on the nanosecond to microsecond time scales while

not explicitly treating the picosecond behavior of the un-

derlying circuit elements.

Here we introduce a phenomenological model of loop

arXiv:2210.09976v1 [cs.NE] 18 Oct 2022

neurons and their constitutive elements that accurately

captures the transfer characteristics of the circuits with-

out solving the underlying circuit equations. Dendrites

are revealed to be central to the system. Each dendrite

is treated with a single, ﬁrst-order ordinary diﬀerential

equation (ODE) that describes the output of the element

as a function of its instantaneous inputs and internal state.

These equations take the form of a leaky integrator with a

nonlinear driving term. The input to each dendrite is ﬂux,

and the output is an integrated current, which is coupled

through a transformer into another dendrite. Synapses are

dendrites with a closed-form expression for the input ﬂux

following each synapse event. The soma of a neuron is also

modeled with the same equations as a dendrite with two

modiﬁcations. First, when the output current reaches a

speciﬁed level, threshold is reached, and an electrical sig-

nal is sent to a transmitter circuit, which produces a pulse

of light. Second, a refractory dendrite triggered oﬀ this

output is coupled back to the soma inhibitively to achieve

a refractory period.

By working at the phenomenological level, the time to

simulate single dendrites is decreased by a factor of ten

thousand while maintaining an accuracy of one part in ten

thousand. The speed advantage grows with the size of the

system being simulated and the duration of the simulation.

The functional form of the speedup with size and duration

has not been fully investigated, as solving the full system

of circuit equations becomes very time consuming for even

small systems. In addition, the model makes transparent

the qualitative behavior of all components of the system,

including the similarities to biological neurons as well as

other physical systems such as Ising models, spin glasses,

and pulse-coupled oscillators.

We begin by motivating the form of the model based

on circuit considerations. We then describe the means by

which the form of the driving term in the leaky integra-

tor is obtained. Error is quantiﬁed by comparison with

full circuit simulations, and convergence is investigated as

a function of time step size. Numerical examples of den-

drites, synapses, and neurons are presented. An example

of a neuron with a dendritic arbor performing an image-

classiﬁcation task is given to illustrate the utility of the

model. Further extensions to enable theoretical treatment

of very large systems are discussed.

2 Overview of Loop Neurons

Research in superconducting optoelectronic networks of

loop neurons aspires to realize artiﬁcial neural systems

with scale and complexity comparable to the human brain.

We have introduced the concepts of loop neurons in a num-

ber of papers [6–11], and we have demonstrated many of

the principles experimentally [12–16]. A schematic dia-

gram of a loop neuron is shown in Fig. 1, depicting the

complex dendritic tree that appears central to the com-

putations of loop neurons [9, 17]. In these neurons, inte-

SiT

Figure 1: Schematic of a loop neuron with an elaborate

dendritic tree. The complex structure consists of excita-

tory and inhibitory synapses (Seand Si) that feed into

dendrites (D). Each dendrite performs computations on

the inputs and communicates the result to other dendrites

for further processing or on to the cell body of the neuron

(N). The neuron cell body acts as the ﬁnal thresholding

stage, and when its threshold is reached, light is produced

by the transmitter (T), which is routed to downstream

synaptic connections via photonic waveguides.

gration, synaptic plasticity, and dendritic processing are

implemented with inductively coupled loops of supercur-

rent. It is due to the prominent role of superconducting

storage loops that we refer to devices of this type as loop

neurons.

Operation of loop neurons is as follows. Photons

from upstream neurons are received by a superconducting

single-photon detector (SPD) at each synapse. Using a su-

perconducting circuit comprising two Josephson junctions

(JJs) coupled to the SPD, synaptic detection events are

converted into an integrated supercurrent which is stored

in a superconducting loop. The amount of current added

to the integration loop during a photon detection event is

determined by the synaptic weight. The synaptic weight is

dynamically adjusted by another circuit combining SPDs

and JJs, and all involved circuits are analog. When the

integrated current of a given neuron reaches a (dynami-

cally variable) threshold, an ampliﬁcation cascade begins,

resulting in the production of light from a waveguide-

integrated semiconductor light emitter. The photons thus

produced fan out through a network of dielectric waveg-

uides and arrive at the synaptic terminals of other neurons

where the process repeats.

The core active component of loop neurons is a circuit

known as a superconducting quantum interference device

(SQUID), which comprises two JJs in parallel. A circuit

diagram is shown in Fig. 2(a). The SQUID is a three-

terminal device with a bias, ground, and an input which

couples ﬂux into the loop formed by the two JJs and in-

R I T

SPD

ispd

RC CI

Jij

SQUID

Vsq

threshold

(a)

(b)

(c)

(d)

Figure 2: Circuit diagrams. (a) A two-junction SQUID that forms the dendritic receiving loop. (b) A dendrite with

receiving (R) and integrating (I) loop. The integrating loops of two other dendrites are input to a collection coil (C)

that delivers ﬂux to the receiving loop. (c) A synapse formed with an SPD input to a dendrite. (d) A soma realized

as a dendrite with thresholding component in the integration loop and initial stage of the transmitter (T).

ductors. For the present purpose, the ﬂux input is the

active signal. When the SQUID is current-biased below

the critical current of the two JJs and the ﬂux input is

below a bias-dependent threshold, the SQUID remains su-

perconducting, and the voltage across the device is zero.

When the applied ﬂux exceeds the threshold at a given

bias point, the JJs will begin to emit a series of voltage

pulses known as ﬂuxons, and the time-averaged voltage

across the device will become non-zero.

To form a dendrite from a SQUID, we ﬁrst ensure that it

is biased below the critical current of the JJs so it is quies-

cent when no ﬂux is applied. The output of the SQUID is

captured by an L-Rloop that performs current integration

with a leak [Fig. 2(b)]. When suﬃcient ﬂux is input to the

SQUID to drive the JJs to begin producing voltage pulses,

the pulses drive current into the L-Rloop, these pulses are

summed in the inductor, and the accumulated signal leaks

with the L/R time constant of the loop. This conﬁgura-

tion of a SQUID coupled to an L-Rloop is referred to as

a dendrite, the SQUID that receives input ﬂux is referred

to as the receiving (R) loop, and the L-Rcomponent is

referred to as the integration (I) loop. The current stored

in the integration loop produces the signal that will be

communicated to other dendrites. To receive signal from

multiple input dendrites, a passive collection coil (C) is

used. A circuit diagram of a dendrite with two inputs, a

collection coil, a receiving loop, and an integrating loop

coupled to an output is shown in Fig. 2(b). All coupling

between dendrites is through magnetic ﬂux communicated

through mutual inductors. The use of transformers for this

purpose mitigates cross talk and enables high fan-in [17].

To form a synapse from a dendrite, we attach an SPD

to the ﬂux input to the receiving loop. This circuit is

shown in Fig. 2(c). When an SPD detects a photon, it

rapidly switches from zero resistance to a large resistance,

diverting the bias current to the other branch of the cir-

cuit. This current is coupled into the receiving loop of the

dendrite as ﬂux, driving the dendrite above threshold and

adding current to the dendrite’s integration loop.

To form a neuron from a dendrite, two modiﬁcations

are required. First, the integration loop must be equipped

with a thresholding element that drives a transmitter cir-

cuit to produce light when the integrated signal reaches

this threshold. This thresholding element coupled to the

transmitter is shown in Fig. 2(d); it is referred to as a tron

and is described in more detail in Sec. 5. Second, an addi-

tional dendrite is attached to the neuron that provides neg-

ative feedback to achieve a refractory period (not shown in

the circuit diagram for simplicity). The refractory period

is a brief period of quiescence following a neuronal spike

event. When the neuron’s integration loop reaches thresh-

old, this refractory dendrite is driven, accumulates signal

in its integration loop, and this signal suppresses the state

of the neuron’s receiving loop. The time constant of the

refractory dendrite establishes the refractory period of the

neuron.

To summarize, a dendrite is a SQUID with a ﬂux input

adding current to a leaky integration loop. A synapse is a

single-photon detector coupled to a dendrite. The soma of

a neuron is a dendrite with a thresholding element in the

integration loop as well as a second dendrite that provides

feedback for refraction. To construct full loop neurons,

many synapses are coupled into a dendritic arbor which

feeds forward into the soma. The output of the soma is

an optical pulse that couples light to a network of waveg-

uides and delivers faint photonic signals to downstream

synapses where they are received with single-photon de-

tectors. This construction is illustrated schematically in

Fig. 1, and the basic circuits are shown in Fig. 2. The cur-

rent in a dendritic integration loop is analogous to the

membrane potential of a biological dendrite [18, 19], and

these signals are the principal dynamical variables of the

system. Inhibitory synapses can be achieved through mu-

tual inductors with the opposite sign of coupling. Complex

arbors with multiple levels of dendritic hierarchy can be

implemented to perform various computations [9] as well

as to facilitate a high degree of fan-in [17].

We see that dendrites are central to loop neurons. The

states of current in all dendritic integration loops speciﬁes

the state of the system. A phenomenological model of loop

neurons must therefore capture the temporal evolution of

these currents as well as the coupling between dendrites.

With these concepts in mind we proceed to construct the

model.

3 Dendrite Model

As mentioned in Sec. 2, a SQUID is the primary active

element of a dendrite. To motivate the phenomenological

dendrite model we require a quantitative understanding of

SQUID operation. The two-junction SQUID of Fig. 2(a)

with symmetrical inductances was modeled using a ﬁrst-

principles circuit model [20], and the results are shown in

Fig. 3. Figure 3(a) shows a time trace of the voltage across

the SQUID when it is in the voltage state. The peaks

corresponding to ﬂuxon production are evident, and the

time-averaged voltage is also shown. While the voltage is

a rapidly varying function of time on the picosecond scale,

the time-averaged voltage is steady.

This behavior is analyzed systematically in Fig. 3(b).

The time-averaged voltage of a symmetric SQUID is plot-

ted as a function of the applied ﬂux for several values

of the bias current, which has been normalized to the

critical current of a single junction (ib=Ib/Ic). The

time-averaged voltage, plotted on the left y-axis, is pro-

portional to the rate of ﬂux-quantum production, plot-

ted on the right y-axis. The relationship results from the

single-valued nature of the superconducting wave function

around the closed SQUID loop, which requires that

Ztfq

Vsq(t)dt = Φ0.(1)

Equation 1 informs us that the time required to produce

a single ﬂuxon, tfq, is related to the voltage across the

squid, Vsq. For constant voltage, we obtain rfq = 1/tfq =

Vsq/Φ0. When viewed over time scales appreciably longer

-1/2 -1/4 0 1/4 1/2

Vsq [μV]

100

Rfq [fluxons per ns]

ib = 1.125

ib = 2.25

Time [ps]

Vsq [μV]

0 20 40 60 80 100

100

150

Vsq

(a)

(b)

Figure 3: The response of a SQUID. (a) Time trace of

a SQUID biased in the voltage state. Fluxon peaks are

marked with crosses of diﬀerent colors for the two JJs.

The time average is also shown as calculated by taking the

average of the time trace between two ﬂuxons produced by

the same junction. (b) The time-averaged voltage across

the SQUID as a function of applied ﬂux, φ, normalized

to the magnetic ﬂux quantum, Φ0, for several values of

normalized bias current, ib.

than tfq, it makes sense to speak of a rate of ﬂux-quantum

production, rfq, and this is the ﬁrst element of our model:

when driven to the active state, a dendrite will begin to

produce ﬂuxons, which carry current, and we can track

this current by monitoring the rate of ﬂuxon production

while ignoring the picosecond dynamics by which the JJs

produce the ﬂuxons.

Several features of Fig. 3(b) are pertinent to the present

study. First, for a given value of ib, a ﬁnite value of ﬂux is

required before the SQUID enters the voltage state. This

provides a non-zero threshold that is relevant to dendritic

computation. This threshold can be adjusted with the

bias current. Second, the response is periodic in applied

ﬂux, with the period being Φ0/2, where Φ0=h/2e≈

2×10−15V·s = 2 mV ·ps is the magnetic ﬂux quantum.

To maintain a monotonic response, the applied ﬂux must

be limited to this value [17]. Third, if the inductors L1

and L2are equal, the response of the SQUID is symmetric

about Φ = 0. These features will be discussed further as

the study proceeds.

The second element of the model captures the integra-

tion and leak of the current generated when the SQUID is

driven above threshold. These behaviors are accomplished

by adding an L-Rloop to the output of the SQUID, as

shown by the integration loop labeled I in Fig. 2(b). The

current integrated in this branch of the circuit is the quan-

tity of interest for the dendrite. It is this quantitiy that

will couple to other dendrites or the neuron cell body, and

it is this quantity we wish to track with our phenomenolog-

ical model. Dendrites comprising a receiving loop coupled

to an integrating loop [Fig. 2(b)] are referred to as RI den-

drites. We know from elementary circuit theory that the

L-Rloop will result in exponential decay of signal with a

time constant of the dendritic integration loop given by

τdi =Ldi/Rdi.

We can now write down a postulated expression for the

signal sstored in the integration loop of a dendrite:

βd s

dτ =r(φ, s;ib)−α s. (2)

Equation 2 states that the signal sgrows in time due to the

driving term, which is the rate of ﬂux quantum produc-

tion, denoted by r. This function rdepends on the applied

ﬂux to the receiving loop of the dendrite, φ= Φ/Φ0, as

well as of the signal present in the integration loop, s. The

rate also depends on the bias current, ib, as a parameter

that throughout this work is assumed to be held constant

over times much longer than the inter-ﬂuxon interval. In

Eq. 2 we have formulated the model in dimensionless units,

where s=Idi/Ic, and Idi is the current present in the

dendritic integration loop. β= 2 π IcLdi/Φ0is a dimen-

sionless parameter that quantiﬁes the inductance of the

loop, and α=Rdi/rj, where rjis the shunt resistance of

each JJ in the resistively and capacitively shunted junc-

tion model [20–23]. The signal sdecays at a rate related

to αand β. Speciﬁcally, the time constant for decay is

given by τdi =Ldi/Rdi =β/ωcα, where ωcis the Joseph-

son characteristic frequency discussed in Appendix A, and

we include the subscript on τdi to refer to the dendritic

integration loop and distinguish that quantity from the

dimensionless time variable entering Eq. 2. Equation 2 is

a leaky integrator ODE. The drive term is the rate of ﬂux

quantum production, and the leak term gives simple ex-

ponential decay, as expected from an L-Rcircuit.

Dendrites are coupled to each other through ﬂux. The

coupling ﬂux from dendrites indexed by jto dendrite iis

given by

φi=

j=1

Jij sj,(3)

where Jij is a coupling term proportional to the mutual

inductance that includes contributions from all the trans-

formers present on the collection coil in Fig. 2(b). Equa-

tion 3 shows that coupling between dendrites is due to the

signal in the integration loop of one dendrite being com-

municated as ﬂux into the receiving loop of a subsequent

dendrite. The signal in the subsequent dendrite is then

obtained through the evolution of Eq. 2 with the ﬂux from

the ﬁrst dendrite providing the ﬂux φin the function r

and the signal from the second dendrite providing the s

term. The ibterm entering rrefers to the bias on the

second dendrite and is treated here as a parameter rather

than a dynamical variable. Further details regarding the

derivation of these expressions is given in Appendix C.

Equations 2 and 3 constitute the phenomenological

model of a dendrite. A neuron or network can be sim-

ulated by solving these coupled equations for all dendrites

in the system. However, we have not speciﬁed the form

for the rate function, r(φ, s;ib), which is central to the

model.

We have arrived at Eq. 2 as a postulate; this expres-

sion is not directly obtained from the underlying circuit

equations. The postulate is that there will be a function

r(φ, s;ib) such that Eq. 2 provides an accurate account of

the signal present in a dendrite’s integration loop under

the circumstances of interest for loop neurons, provided

we only inquire about the signal over time scales appre-

ciably longer than the inter-ﬂuxon interval, which is on

the order of 10 ps. We aim to interrogate dendrites on

time scales of 100 ps or longer, with neuron and network

activity of interest on time scales from nanoseconds to the

longest timescales that can be simulated under the limits

of computational resources.

Our procedure for obtaining r(φ, s;ib) is as follows. A

dendrite with a SQUID as a receiving loop and an L-

Rbranch as an integrating loop is numerically modeled

with the circuit equations given in Appendix A. A con-

stant value of ﬂux is applied to the receiving loop, and the

rate of ﬂux-quantum production is monitored as a func-

tion of time while current accumulates in the integration

loop. For these simulations, the resistance of the integra-

tion loop is set to zero. This procedure is repeated for

many values of φand ibto obtain what we refer to as

“rate arrays”, which are shown in Fig. 4, where r(φ, s;ib)

is plotted as a function of φand sfor three values of ib.

Here we work in dimensionless units, so the units of r

are ﬂuxons generated per unit of dimensionless time, τ,

which is related to the JJ characteristic frequency (Ap-

pendix A). It can be seen that the value of r(φ, s;ib) is

monotonically increasing with φover the range considered

here, while accumulation of sdecreases the rate of ﬂux-

quantum production. This decrease is because addition of

current to the integration loop diverts the bias away from

the SQUID, so the voltage is decreased, and the rate is

reduced in accordance with Eq. 1. When suﬃcient signal

is accumulated in the integration loop, the rate of ﬂux

quantum production drops to zero, and we say the loop is

saturated.

For a small value of ib[Fig. 4(a)] a large amount of ﬂux is

required to drive the dendrite above threshold to the active

state, and a small signal sis present at saturation. As ibis

increased [Figs. 4(b) and (c)] the threshold is reduced, and

the saturation level is increased. The qualitative shape of

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

PhenomenologicalModelofSuperconductingOptoelectronicLoopNeuronsJereyM.Shainline,BryceA.Primavera,andSaeedKhanNationalInstituteofStandardsandTechnology325Broadway,Boulder,CO,USA,80305jerey.shainline@nist.govOctober17,2022AbstractSuperconductingoptoelectronicloopneuronsareaclassofcircuitspotential...

展开>> 收起<<

Phenomenological Model of Superconducting Optoelectronic Loop Neurons Jerey M. Shainline Bryce A. Primavera and Saeed Khan.pdf

共34页,预览5页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Phenomenological Model of Superconducting Optoelectronic Loop Neurons Jerey M. Shainline Bryce A. Primavera and Saeed Khan

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: