Note: exercize one id the follwoing: Derive that the gradient of the cross entropy loss function used in binary logistic regression takes the form:

????=1/????∑????????=1[(????????^−????????)∗????????]

Transcribed Image Text:

4. Stochastic Gradient. In this exercise, we will design and perform a small numerical experiment to verify that the expected value of the stochastic gradient is the true gradient. Consider the setup where we are given the MNIST data and our goal is to write a binary logistic classifier the predict whether a given image is the digit 5 (label "1") or not (label "0"). We start our learning from a random point in parameter space (say, generated from a normal distribution using torch.randn) using the result of Exercise 1, compute the gradient of the loss J at that point. using pytorch, write an expression for the loss J and compute its gradient at that point using backward (). Do you get the same answer? ● using a batch size of b = 32, write a function that returns a stochastic gradient of J by choosing b randomly chosen images from the dataset. • call the stochastic gradient function a large number of times to obtain an estimate of its expected value. Compare with the full gradient. 4. Stochastic Gradient. In this exercise, we will design and perform a small numerical experiment to verify that the expected value of the stochastic gradient is the true gradient. Consider the setup where we are given the MNIST data and our goal is to write a binary logistic classifier the predict whether a given image is the digit 5 (label "1") or not (label "0"). We start our learning from a random point in parameter space (say, generated from a normal distribution using torch.randn) using the result of Exercise 1, compute the gradient of the loss J at that point. using pytorch, write an expression for the loss J and compute its gradient at that point using backward (). Do you get the same answer? ● using a batch size of b = 32, write a function that returns a stochastic gradient of J by choosing b randomly chosen images from the dataset. • call the stochastic gradient function a large number of times to obtain an estimate of its expected value. Compare with the full gradient.

Answer rating: 100% (QA)

The complete a... View the full answer