Solution to Mathematics for Machine Learning Exercise 4.12
Solution: By SVD (Singular Value Decomposition), we have $A=U\Sigma V^T$. Let $y=V^T x$, then \[\|y\|_2^2=y^Ty=(V^Tx)^TV^Tx=x^TVV^Tx=x^Tx=\|x\|_2^2.\]Then we have\begin{align*}\|A x\|_2^2=&\ (Ax)^T(Ax)=x^TA^TAx\\ = &\ x^T V \begin{bmatrix}\sigma_1^2 & 0 & 0\\ 0 &…