Q1) Discriminant Linear Classifiers:

You are given a training data set {xn, tn} of size N = 21. Each input vector xn is a point in the 2-dimensional Euclidean space R2 . We have x1 = (0, 0), x2 = (1, 0), x3 = (2, 0), x4 = (0, 1), x5 = (1, 1), x6 = (2, 1), x7 = (3, 1), x8 = (4, 1), x9 = (5, 1), x10 = (100, 1), x11 = (0, 2), x12 = (1, 2), x13 = (2, 2), x14 = (3, 2), x15 = (4, 2), x16 = (5, 2), x17 = (100, 2), x18 = (3, 3), x19 = (4, 3), x20 = (5, 3), and x21 = (100, 3).]

There are two target classes C1 and C2. For each point xn in the training set, xn belongs to C1 if its second coordinate is less than or equal to 2, and belongs to C2 otherwise. If 1 xn C1, we have tn = 1. If xn C2, we have tn = 0 in the questions regarding least-squares linear discriminant and Fisher's linear discriminant, and have tn = 1 in the question on the perceptron algorithm.

(A) Compute the least-square linear classifier based on the training data. You need to write out (a) the error function, (b) the computed parameters (w0, w1, w2), and (c) plot the classification together with the training data.

(B) Compute the linear classifier based on the training data using Fisher's linear discriminant. You need to write out (a) the error function, (b) the computed parameters (w0, w1, w2), and (c) plot the classification together with the training data.

(C) Compute the linear classifier based on the training data using the perceptron algorithm, starting with the initial parameter (w0, w1, w2) = (1.5, 0, 0). For each iteration, you need to specify (a) the iteration number, (b) the current parameters, (c) the mis-classified input xn used in that particular iteration of stochastic gradient descent, and (d) the updating vector. When the algorithm converges, plot the classification together with the training data.

Q2). Continuous Bayes Classifier:

We want to build a Bayes classifier for a binary classification task (y = 1 or y = 2) with a 1-dimensional input feature (x). We know the following quantities: (1) P(y = 1) = 0.6; (2) P(x|y = 1) = 0.5 for 0 x 2 and P(x|y = 1) = 0 otherwise; and (3) P(x|y = 2) = 0.125 for 0 x 8 and P(x|y = 2) = 0 otherwise.

(A) What is the prior for class label y = 2?

(B) What is P(y = 1|x)?

(C) For x = 1, what is the class label your classifier will assign? Why? What is the risk of this decision?

(D) What is the decision boundary of your Bayes classifier?

Q3). Discrete Bayes Classifier :

We want to build a Bayes classifier for a binary classification task (y = 1 or y = 2) with two binary features (x1 and x2). We know the following quantities: (1) P(y = 1) = 0.6; (2) P(x1 = 0, x2 = 0|y = 1) = 0.3, P(x1 = 0, x2 = 1|y = 1) = 0.1, P(x1 = 1, x2 = 0|y = 1) = 0.4, P(x1 = 1, x2 = 1|y = 1) = 0.2, and (3) P(x1 = 0, x2 = 0|y = 2) = 0.4, P(x1 = 0, x2 = 1|y = 2) = 0.3, P(x1 = 1, x2 = 0|y = 2) = 0.2, P(x1 = 1, x2 = 1|y = 2) = 0.1, 2

(A) What is the prior for class label y = 2?

(B) What is P(y = 1|x)?

(C) For an example with x1 = 0 and x2 = 1, what is the class label your classifier will assign? Why? What is the risk of this decision?

(D) What is the decision boundary of your Bayes classifier?