Consider a binary classification problem of finding the binary labels y in $\{1,1\},$ for input examples of the

form x in R

d$\backslash$times $1$

$.$ We will use the following loss function which is based on margin m $=$ sy $=$

wT x

y

L$($m$,$ w$)=($

$0\mathrm{.}5$m$2$

for m $<=0$

$|$m$|$ Otherwise

a$.$ For the case of d $=2$ i$.$e$.$ x $=[$x$1$ x$2]$

T

$,$ find the gradient of loss function $$wL$($m$,$ w$)$ w$.$r$.$t$.$ unknown weight vector

w $=[$w$1$ w$2]$

T

$.$ You may compute $$

$$w$1$

L$($m$,$ w$),$ and $$

$$w$2$

L$($m$.$w$)$ and then stack them into the required gradient vector.

Please also note the following derivative rule that you may need: $[6]$

$$

$$w $|$f$($w$)|=$

f$($w$)$

$|$f$($w$)|$

$$

$$w f$($w$)$

b$.$ Now, assume following training data having N $=4$ examples, is available: $[4]$

i Input: xi Output: yi

$1$ x$1=[02]$T y$1=1$

$2$ x$2=[01]$T y$2=1$

$3$ x$3=[10]$T y$3=1$

$4$ x$4=[10]$T y$4=1$

For the given loss function L$($m$,$ w$),$ and given training dataset, write down the average loss in terms of unknown weight vector

w $=[$w$1$ w$2]$

T

as

Lavg$($w$)=1$

N

XN

i$=1$

L$($mi$,$ w$)$

Compute the gradient vector of average loss Lavg$($w$)$ i$.$e$.$wLavg$($w$)$ w$.$r$.$t$.$ unknown weight vector w $=[$w$1$ w$2]$

T

$.$

c$.$ Starting from an initial weight vector w$(0)=[00\mathrm{.}5]$T

$,$ run single iteration of gradient descent algorithm to find w$(1)$ with

step size $\backslash$alpha $=0\mathrm{.}2[2]$

d$.$ Report the classification accuracy on the provided training data if you decide to use w$(1)$ as your final weights for the model.

You can assume sign$(.)$ function as your activation function