****HW2 (question and solution) for the dataset. In HW2 the dataset used is from HW1 both the questions are listed below. (question and solution). All the HWs are linked. there is no missing information as such, all the information has been provided... also for reference there is a link to the youtube video.

Use Keras Model with three layers of neural network to classify the dataset in HW#2 (for both questions). The first layer (input layer) has four neurons, the second layer has three neurons, and the third layer (output layer) has only one neuron. You should use at least two different activation function and two different optimization functions.

Explain your experience with this network in compare with the ones you did in HW#2 and also the effect of different activation and optimization functions.

Hint: Go over the following video

https://www.youtube.com/watch?v=bemDFpNooA8&ab_channel=TensorFlow

*****************HW2*******************

Question :

Use a fully connected two layer neural network to classify the same dataset of Hw#1 (for both questions). The first layer (input layer) has four neurons and the second layer (output layer) has only one neuron. You should use the sigmoid activation function and the same learning rules that was discussed in the class. Please print the input, Ws, and output values after training. Also explain your experience with this network in compare with the one you did in HW# 1

Solution :

Let us now implement the perceptron algorithm from scratch in python using only numpy as an external library for matrix-vector operations. We will implement it as a class that has an interface similar to other classifiers in common machine learning packages like Sci-kit Learn. We will implement for this class 3 methods: .fit(), .predict(), and .score()

The .fit() method will be used for training a perceptron. It expects as the first parameter a 2D numpy array X. The rows of this array are samples from our dataset, and the columns are the features. The second parameter,y, should be a 1D numpy array that contains the labels for each row of data in X. The third parameter, n_iter, is the number of iterations for which we let the algorithm run.

def fit(self, X, y,n_iter=100):

n_samples = X.shape[0]

n_features = X.shape[1]

self.weights = np.zeros((n_features+1,))

X = np.concatenate([X,np.ones((n_samples, 1))], axis=1)

for i in range(n_iter):

for j in range(n_samples):

if y[j]*np.dot(self.weights, X[j, :]) <= 0:

self.weights += y[j]*X[j, :]

The .predict() method will be used for predicting labels of new data. It first checks if the weight object attribute exists, if not this means that the perceptron is not trained yet. We use np.vectorize() to apply this mapping to all elements in the resulting vector of matrix multiplication.

def predict(self, X):

if not hasattr(self, 'weights'):

print('The model is not trained yet!')

return

n_samples = X.shape[0]

x=np.concatenate([X, np.ones((n_samples, 1))],axis = 1)

y=np.matul(X, self.weights)

y=np.vectorize(lambda val: 1 if val > 0 else -1 )(y)

return y

The score () method is used to compute the accuracy of the predictions.

def score(self, X, y):

pred_y = self.predict(X)

return np.mean(y==pred_y)

#Here is the code I have written for the above scenario

import numpy as np class Perceptron: def fit(self, X, y, n_iter=100): n_samples = X.shape[0] n_features = X.shape[1] # Add 1 for the bias term self.weights = np.zeros((n_features+1,)) # Add column of 1s X = np.concatenate([X, np.ones((n_samples, 1))], axis=1) for i in range(n_iter): for j in range(n_samples): if y[j]*np.dot(self.weights, X[j, :]) <= 0: self.weights += y[j]*X[j, :] def predict(self, X): if not hasattr(self, 'weights'): print('The model is not trained yet!') return n_samples = X.shape[0] # Add column of 1s X = np.concatenate([X, np.ones((n_samples, 1))], axis=1) y = np.matmul(X, self.weights) y = np.vectorize(lambda val: 1 if val > 0 else -1)(y) return y def score(self, X, y): pred_y = self.predict(X) return np.mean(y == pred_y) view raw

*************HW1***************

Question :

1. Use Jupyter to train a simple perceptron model to classify the patterns on a NAND function. To begin training, select your initial weights randomly between 2 and -2.

X1	X2	Target
0	0	1
0	1	1
1	0	1
1	1	0

2. Use Jupyter to train a simple perceptron model to classify a set of randomly generated patterns. Generate randomly two set of data set, each one with 50 data patterns, in two dimensions x1 and x One data set has 0 <= x1 <= 3 and 0 <= x2 <= 3 with target of 0, and the second data set has 6 <= x1 <= 9 and 6 <= x2 <= 9 with target of 1. To begin training, select your initial weights randomly between 2 and -2

Solution :

import numpy as np import matplotlib.pyplot as plt %matplotlib inline

NUM_FEATURES = 2 Max_Iter = 100 learning_rate = 0.7

x = np.array([[0, 0], [1, 0], [1, 1], [0, 1]]) # training input features y = np.array([1, 1, 1, 0]) # training labels for OR operation

W = np.random.randint(-2,2,size=(2)) b = np.random.rand(1)

N, d = np.shape(x) # number of samples and number of features

for k in range(Max_Iter): for j in range(N): yHat_j = x[j, :].dot(W) + b yHat_j = 1.0 / (1.0 + np.exp(-yHat_j)) err = y[j] - yHat_j

deltaW = err * x[j, :] deltaB = err W = W + learning_rate * deltaW b = b + learning_rate * deltaB # print('W:' + str(W)) #print weight and threshold term after each iteration # print('b:' + str(b)) # Now plot the fitted line. We need only two points to plot the line plot_x = np.array([np.min(x[:, 0] - 0.2), np.max(x[:, 1]+0.2)]) plot_y = - 1 / W[1] * (W[0] * plot_x + b) # comes from, w0*x + w1*y + b = 0 then y = (-1/w1) (w0*x + b)

plt.scatter(x[:, 0], x[:, 1])#, c=y, s=100, cmap='viridis') plt.xlim([-0.2, 1.2]); plt.ylim([-0.2, 1.25]); plt.plot(plot_x, plot_y, color='k', linewidth=2) plt.show() activation = x.dot(W) + b loss = (activation > 0)!=y if np.sum(loss)==0: break

print("Final values of weights & threshold: ") print('W:' + str(W)) print('b:' + str(b))

#testing activation = x.dot(W) + b

print("Testing: ") print("Input \t: Ouput") for i in range(4): if (activation[i] > 0.0): print(x[i]," : ",1) else: print(x[i]," : ",0)