When X does not have full rank, let’s see why PX = X(XTX)

−XT is invariant to the choice of generalized inverse. Let G and H be two generalized inverses of XTX. For an arbitrary v ∈ Rn, let v = v1 + v2 with v1 = Xb ∈ C(X) for some b.

a. Show that vTXGXTX = vTX, so that XGXTX = X for any generalized inverse.

b. Show that XGXTv = XHXTv, and thus XGXT is invariant to the choice of generalized inverse.