Question: Is this implementation of stochastic BFGS correct? def s _ bfgs ( w _ 0 , B _ 0 , data, labels, pred _ f

Is this implementation of stochastic BFGS correct?
def s_bfgs(w_0, B_0, data, labels, pred_f=prediction, grad_f=stochastic_gradient, loss_f=logloss, max_iter=100, tol=0.0001, batch_size=50):
w = w_0
B_inv = np.linalg.inv(B_0)
grad_w = grad_f(w, data, labels, batch_size)
for i in range(max_iter):
p =-np.dot(B_inv, grad_w) # Use np.dot instead of np.outer
alpha = wolfe(w, p, data, labels)
s
= alpha * p
w_new = w + s
grad_new = grad_f(w_new, data, labels, batch_size)
if np.linalg.norm(grad_new - grad_w)< tol:
break
y = grad_new - grad_w
grad_w = grad_new
sy = np.dot(s, y)
B_inv = B_inv +(1+ np.dot(y.T, np.dot(B_inv, y))/ sy)* np.outer(s, s)/ sy -(np.outer(np.dot(B_inv, y), s)+ np.outer(s, np.dot(B_inv, y)))/ sy
predictions = pred_f(w, data)
loss = loss_f(predictions, labels)
print("iter:", i," loss:", loss)
w = w_new
grad_w = grad_new
return w

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Databases Questions!