Question: In this problem you need to show that when the two classes are linearly separable, the perceptron algorithm will converge. Specifically, for a binary classification

 In this problem you need to show that when the two

In this problem you need to show that when the two classes are linearly separable, the perceptron algorithm will converge. Specifically, for a binary classification dataset of N data points, where every xi has a corresponding label yi{1,1} and is normalized: xi=xiTxi=1,i{1,2,,N}, the perceptron algorithm proceeds as below: In other words, weights are updated right after the perceptron makes a mistake (weights remain unchanged if the perceptron makes no mistakes). Let the (classification) margin for a hyperplane w be (w)=mini[N]wwTxi (convince yourself that (w) is the smallest distance of any data point from the hyperplane). Let wopt be the optimal hyperplane, i.e. it linearly separates the classes with maximum margin. Note that since data is linearly separable there will always exist some wopt. Let =(wopt). Following the steps below, you will show that the perceptron algorithm makes a finite number of mistakes that is at most 2, and therefore the algorithm must converge. 3.1 Show that if the algorithm makes a mistake, the update rule moves it towards the direction of the optimal weights wopt. Specifically, denoting explicitly the updating iteration index by k, the current weight vector by wk, and the updated weight vector by wk+1, show that, if yiwkTxi

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Databases Questions!