Implement the NAIVEBAYES algorithm

Consider a binary classification problem, where the input $x$ is a binary vector of length $k.$ Naive Bayes is a

generative model that assumes that features in $x$ are independent given $y,$ and models each feature with a

Bernoulli distribution, $p(x|y)=pro{d}_{i}p(x_i|y).$ Note that the features are not identically distributed.

Using the training data, the algorithm estimates the model parameters by maximixing the log$-$likelihood.

Then given a new data point $x,$ the algorithm selects the label that has the highest probability

$logp(y|x)=logp(y)+logp(x|y)-logp(x).$

Note that we can ignore $p(x)$ since it is constant.

This classifier is a linear classifier that is$,$ one can write it as sign$(w^{T}x+b).$

Implement $,$ labels$)$ that computes the weight vector $w$ and $b$ given the training data.

Hints: The model has $2k+1$ parameters, figure out what they are, and what are their ML estimates. Once

you figure out the estimates, derive the weights $w$ and $b$ from these estimates. A convenient way of expressing

probability mass function for a single Bernoulli random variable is $p(a)={}^a(1-{)}^{1-a},$ where $a=0,1.$ You have to fill the below python code according to the above and given instructions. Note that it corrects using another code. I will attach the correcting algorithmic code too. Answer: "import sys

import numpy as np

def nb$($X$,$ labels$)$:

$"""$

Computes the weight vector w and and bias b corresponding

to the Naive Bayes classifier with Bernoulli components.

Parameters

$----------$

X : an array of size $($n$,$ k$)$

training input data for the classifier, elements must be $0/1$

labels : an array of size n

training labels for the classifier, elements must be $0/1$

Returns

$-------$

w : an array of size k

weights corresponding to the classifier

bias: real number

bias term corresponding to the classifier

$"""$

cnt$,$ k $=$ X$.$shape

w $=$ np$.$zeros$($k$)$

b $=0$

# place your code here

return w$,$ b

def main$($argv$)$:

D $=$ np$.$loadtxt$($argv$[1])$

X $=$ D$[$:$,1$:$]$

labels $=$ D$[$:$,0]$

print$($nb$($X$,$ labels$))$

# This allows the script to be used as a module and as a standalone program

if $\_\_$name$\_\_=="\_\_$main$\_\_"$:

if len$($sys$.$argv$)!=2$:

print$(\text{'}$usage: python $\%$s filename' $\%$ sys$.$argv$[0])$

else:

main$($sys$.$argv$)"$ Correcting code: "import numpy as np

from $.$utils import errormsg, ApproxTest

@points$(\text{'}2\mathrm{.}1\mathrm{.}1\text{'},\text{'}2\mathrm{.}1\mathrm{.}2\text{'},\text{'}2\mathrm{.}1\mathrm{.}3\text{'},\text{'}2\mathrm{.}1\mathrm{.}4\text{'},\text{'}2\mathrm{.}1\mathrm{.}5\text{'})$

class NbTester$($ApproxTest$)$:

def test$\_$nb$($self$)$:

D $=$ np$.$loadtxt$(\text{'}$test$/$toy$.$txt$\text{'})$

X $=$ D$[$:$,1$:$]$

labels $=$ D$[$:$,0]$

nb $=$ load$(\text{'}$src$.$nb$\text{'},\text{'}$nb$\text{'})$

w$,$ b $=$ nb$($X$,$ labels$)$

w$1=$ np$.$array$([-1\mathrm{.}5040774,1\mathrm{.}38629436,0\mathrm{.}69314718])$

b$1=-0\mathrm{.}040821994520255256$

self.assertApprox$($w$,$ w$1,$ errormsg$("$Incorrect weight vector $($input toy.txt$)",$ w$1,$ w$))$

self.assertApprox$($b$,$ b$1,$ errormsg$("$Incorrect bias $($input toy.txt$)",$ b$1,$ b$))"$