Problem 4: Least Squares Invertibility (4 points) The following proposition was given in lecture without proof. Suppose we have a dataset of m examples, each with d features, x1, . . . , xm ? Rd. We know that satisfying the following equality is equivalent to satisfying the least squares condition. Aw = b, where A = m? i=1 xixT i and b = m? i=1 yixi. This can be expressed more directly in matrix form by defining A = ... ... ... x1 xm ... ... ... ... ... ... x1 xm ... ... ... T , and b = ... ... ... x1 xm ... ... ... y1 ... ym . Show that A is invertible if and only if x1, xm span Rd

Problem 4: Least Squares Invertibility (4 points) The following proposition was given in lecture without proof. Suppose we have a dataset of m examples, each with d features, X1, ..., Xm E Rd. We know that satisfying the following equality is equivalent to satisfying the least squares condition. m Aw = b, where A = ) x x] and b = >yixi. i= 1 i= 1 This can be expressed more directly in matrix form by defining T yl A = X1 Xm X1 Xm and b= X1 Xm . . . ym Show that A is invertible if and only if X1, . . . Xm span Rd. Note: If you're new to proofwriting or unsure of what constitutes a complete argument, don't hesitate to ask questions