Homework $2$

Part $1$: Perceptron from Scratch

assignment, you will have a working perceptron model that can classify linearly separable datasets.

Background

inputs and "fires" or "activates" if the sum exceeds a threshold.

The perceptron decision boundary is defined by a weight vector $w$ and a bias $b.$ The algorithm updates these parameters based on the training data.

Given a training set of $n$ samples $(x_1,y_1),(x_2,y_2),$dots,$(x_n,y_n),$ where $x_{i}in{R}^d$ and $y_{i}in\{-1,1\},$ the perceptron makes predictions using the sign of the linear combination of the inputs and weights:

hat$(y)(x)=$sign$(w*x+b)$

where sign$(z)$ is a function that returns $1$ if $z0$ and $-1$ otherwise.

The perceptron updates its weights $w$ and bias $b$ using the perceptron learning rule:

$w_{\text{n}\text{e}\text{x}\text{t}\text{s}\text{t}\text{e}\text{p}}=w+(y-$hat$(y))x$

$b_{\text{n}\text{e}\text{x}\text{t}\text{s}\text{t}\text{e}\text{p}}=b+(y-$hat$(y))$

Here, $$ is the learning rate, a small positive scalar determining the step size of the update, $y$ is the true label, and hat$(y)$ is the predicted label.

Algorithm Steps

Initialization: Start with $w=0$ or small random values and $b=0.$

For each training example $(x_i,y_i)$ :

$$ Compute the output hat$(y)(x_i)=$sign$(w*x_i+b).$

Update the weights and bias if hat$(y)y_i$ :

wlarrw$+(y_i-(hat(y))(x_i))x_i$

blarrb$+(y_i-(hat(y))(x_i))$

Repeat: Go through the training set multiple times, until the algorithm converges $($no misclassifications$)$ or a predefined number of iterations is reached.

The perceptron algorithm is guaranteed to converge on linearly separable data sets, but it does not converge if the data is not linearly separable.

Your Implementation

$1\mathrm{.}1$: Define the Perceptron Class: Create a class Perceptron with methods for initialization $(\_),$ training $($fit$),$ and prediction $($pred ict$).$

The$\_\_$init$\_\_$ method should initialize the weights and the threshold.

The fit method should implement the perceptron learning algorithm, adjusting the weights based on the inputs and the target output.

The predict method should take inputs and produce the perceptron's output for those inputs.

$1\mathrm{.}2$: Implement the Learning Rule

Within the fit method, implement the learning rule:

$w_i=w_i+w_i$

where:

$w_i=(y-$hat$(y))x_i$

$i^{th}$ feature of the input vector $x.$

a feature based on how influential that feature is in causing the error.

$1\mathrm{.}3$: Testing and Evaluation

testing sets, using an $\frac{80}{20}$ split.

You might use the make$\_$classification or make$\_$blobs functions from the sklearn. datasets module to generate this dataset.

Plot your dataset to visualize the distribution of the classes. Create a scatterplot, with axes representing the two features and different colors for each class.

Evaluate Your Model: After training, evaluate the accuracy of your model on the testing set. Calculate the proportion of correctly predicted instances to demonstrate the efficacy of your perceptron.