Below is a very simple data set encoding an XOR pattern. + INPUT 1 0 0...

Transcribed Image Text:

Below is a very simple data set encoding an XOR pattern. + INPUT 1 0 0 1 1 INPUT 2 0 1 0 1 OUTPUT weights of each of the links, assuming a learning rate of 0 1 1 0 (a) Draw a diagram of an ANN's topology that can learn this pattern based on the given data, using a single hidden layer with three units. Name all relevant elements. Initialise all weights to 1. (b) For a given machine learning problem, explain the relation between the sample distribution and the true distribution of that problem. (c) For this problem, how can you sensibly increase the set of data? Explain the nature of this dataset in terms of how it represents the underlying distribution from which it's sampled. (d) Draw a single ANN unit ('neuron'), clearly indicating inputs, outputs, and clearly segregating linear combinations of signals from the non-linear components. Assign appropriate variable and function names and include these in your diagram. (e) Assume the units have no non-linear activation function, and that you will use stochastic (online) gradient descent working through the data in sequence. Calculate backpropagation for the first two data points of Table 1 and the topology as described in (a) above. For both rounds, give (i) the errors for both input and output units, (ii) the error derivatives and (iii) the weight updates and (iv) new η = 0.1 Below is a very simple data set encoding an XOR pattern. + INPUT 1 0 0 1 1 INPUT 2 0 1 0 1 OUTPUT weights of each of the links, assuming a learning rate of 0 1 1 0 (a) Draw a diagram of an ANN's topology that can learn this pattern based on the given data, using a single hidden layer with three units. Name all relevant elements. Initialise all weights to 1. (b) For a given machine learning problem, explain the relation between the sample distribution and the true distribution of that problem. (c) For this problem, how can you sensibly increase the set of data? Explain the nature of this dataset in terms of how it represents the underlying distribution from which it's sampled. (d) Draw a single ANN unit ('neuron'), clearly indicating inputs, outputs, and clearly segregating linear combinations of signals from the non-linear components. Assign appropriate variable and function names and include these in your diagram. (e) Assume the units have no non-linear activation function, and that you will use stochastic (online) gradient descent working through the data in sequence. Calculate backpropagation for the first two data points of Table 1 and the topology as described in (a) above. For both rounds, give (i) the errors for both input and output units, (ii) the error derivatives and (iii) the weight updates and (iv) new η = 0.1