Question: In this problem you need to show that when the two classes are linearly separable, the perceptron algorithm will converge. Specifically, for a binary classification

In this problem you need to show that when the two classes are linearly separable, the perceptron algorithm will converge. Specifically, for a binary classification dataset of N data points, where every xi has a corresponding label yi{1,1} and is normalized: xi=xiTxi=1,i{1,2,,N}, the perceptron algorithm proceeds as below: In other words, weights are updated right after the perceptron makes a mistake (weights remain unchanged if the perceptron makes no mistakes). Let the (classification) margin for a hyperplane w be (w)=mini[N]wwTxi (convince yourself that (w) is the smallest distance of any data point from the hyperplane). Let wopt be the optimal hyperplane, i.e. it linearly separates the classes with maximum margin. Note that since data is linearly separable there will always exist some wopt. Let =(wopt). Following the steps below, you will show that the perceptron algorithm makes a finite number of mistakes that is at most 2, and therefore the algorithm must converge. 3.1 Show that if the algorithm makes a mistake, the update rule moves it towards the direction of the optimal weights wopt. Specifically, denoting explicitly the updating iteration index by k, the current weight vector by wk, and the updated weight vector by wk+1, show that, if yiwkTxi
Step by Step Solution
There are 3 Steps involved in it
Get step-by-step solutions from verified subject matter experts
