Question: Let's start by doing gradient descent by hand. For this, you cannont use any python packages w you should do this using your algebra skills
Let's start by doing gradient descent by hand. For this, you cannont use any python packages w you should do this using your algebra skills and a calculator, I recommen doing this on pen and paper and then attaching a picture/scan of your work to this python notebook Part 1: Only Estimate Slope For this example, let's assume we are only trying to estimate one variable given one data point. Let's just consider the slope m of a line and one data point x = 1 . For gradient descent, we'll also need a couple of parameters set ahead of time: 1) starting slope, 2) the cost function (and its partial derivative writ the parameter we are trying to estimate), and 3) the step size/learning rate. Let's start with m=3, a cost function cost(m)=m3, and a step size/learning rate =0.5 Using some calculus, the partial derivative of the cost function wr.t. m is sdron=3m2. This derivative is what you will use to calculate the gradient. 1. Using these starting variables and a starting data point of x=1, what is the slope after the first iteration of gradient descent? (HINT: it should be less than 3 ) 2. What is updated slope on the next iteration? and the third iteration? 3. What is happening to the slope after each iteration? is it increasing, decreasing, staying the same
Step by Step Solution
There are 3 Steps involved in it
Get step-by-step solutions from verified subject matter experts
