Question: I do not know how to approach this problem, any insight would be greatly appreciated! In this problem, we'll compute the posterior distribution for a

I do not know how to approach this problem, any insight would be greatly appreciated!

I do not know how to approach this problem, any
In this problem, we'll compute the posterior distribution for a Gaussian linear regression model, and explore the relationship between the maximum a posteriori (MAP) estimate and the result of Ridge regression. You may nd it helpful to review Chapter 18 of the Data 100 textbook, which covers multiple linear regression. Recall that in linear regression, we're trying to predict a (scalar) number 3; using a vector of (xed) features :1: 6 Rd. Suppose we have n data points (pairs of X and y). We can write inTxi+5a i=1:---:Th (1) where each 5,- N N (0, 02) are independent of each other, and B 6 Rd and 02 > 0 are unknown. Our goal is to estimate 16' , which tells us about the relationship between X and 3;. Let y = (3,1,. . . ,yn) be the vector of y-values; 5 = (51, . . . ,5\") be the vector of 8 values, and let X denote the n x d matrix whose ith row is equal to 3,. Using this notation and linear algebra, we can write the model as y=X+E, 5~N(0102I-)1 (2) where In denotes the n x 11 identity matrix. (a) (3 points) We'll think about the problem from a Bayesian perspective and assume that [3 is random. We put a zero-mean normal prior on [3, Le. ,8 N N (0,0313). To simplify the algebra, we'll also assume that 0' = 1 so that 5 ~ N(0, In). Show that the posterior distribution for ,8 given the data 3; satises p(ly) oc eXp {gm} . (3) where q(5) = TA bTB l c, where A is a d x d square matrix, and b 6 Rd is a vector. What are A and b (you may ignore c)? You must Show your work! Hint: use Bayes' rule and linear algebra

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Mathematics Questions!