If u(k) is an approximation to the solution to A u = b, then the residual vector r(k) = b - Au(k) measures how accurately the approximation solves the system.
(a) Show that the Jacobi iteration can be written in the form u(k+l) = u(k) + D-1r(k)
(b) Show that the Gauss-Seidel iteration has the form u(k+l) = u(k) + (L + D)-1r(k)
(c) Show that the SOR iteration has the form u(k+l) = u(k) + (ωL + D)-1r(k)
(d) If ||r(k)|| is small, does this mean that u(k) is close to the solution? Explain your answer and illustrate with a couple of examples.