A popular construct in 'artificial intelligence' is the recurrent neural network (RNN). RNNs are high-dimensional dynamical...

Transcribed Image Text:

A popular construct in 'artificial intelligence' is the recurrent neural network (RNN). RNNs are high-dimensional dynamical systems. Here is one possible model of a 'vanilla' RNN in continuous time: +tanhΣ Wijxj (1) where i = 1, 2, ..., n, x; ER and wij is a scalar parameter governing 'connections' between states/neurons. There are many uses of RNNs, both practical and theoretical. One way that RNNs are conceptualized are as 'memory networks,', where each equilibrium point, an invariant configuration in state space, is a 'memory'. 1. Provide observations regarding the number of such 'memories' that are encoded in any given RNN of the form (1). 2. Suppose you have a specific 'memory', x* that you are interested in. What are some potentially relevant stability properties of x* and how would you characterize them as completely and specifically as possible? A popular construct in 'artificial intelligence' is the recurrent neural network (RNN). RNNs are high-dimensional dynamical systems. Here is one possible model of a 'vanilla' RNN in continuous time: +tanhΣ Wijxj (1) where i = 1, 2, ..., n, x; ER and wij is a scalar parameter governing 'connections' between states/neurons. There are many uses of RNNs, both practical and theoretical. One way that RNNs are conceptualized are as 'memory networks,', where each equilibrium point, an invariant configuration in state space, is a 'memory'. 1. Provide observations regarding the number of such 'memories' that are encoded in any given RNN of the form (1). 2. Suppose you have a specific 'memory', x* that you are interested in. What are some potentially relevant stability properties of x* and how would you characterize them as completely and specifically as possible?

Answer rating: 100% (QA)

The quantity of memories stored in an RNN Several factors including the size of the state space n the number of neurons the connection pattern describ... View the full answer