Solution to Mathematics for Machine Learning Exercise 2.6
$ \newcommand{\colvec}[1]{\begin{bmatrix} #1 \end{bmatrix}} \newcommand{\rowvec}[1]{\begin{bmatrix} #1 \end{bmatrix}} \newcommand{\mat}[1]{{\boldsymbol{{#1}}}} % matrix \renewcommand{\vec}[1]{{\boldsymbol{{#1}}}} % vector $ Using Gaussian elimination, find all solutions of the inhomogeneous equation system $\mat A \vec x =…
Solution to Mathematics for Machine Learning Exercise 7.7
Consider the quadratic program illustrated in Figure 7.4, $$ \min _{\boldsymbol{x} \in \mathbb{R}^{2}}\quad \frac{1}{2}\left[\begin{array}{l} x_{1} \\ x_{2} \end{array}\right]^{\top}\left[\begin{array}{ll} 2 & 1 \\ 1 & 4 \end{array}\right]\left[\begin{array}{l} x_{1} \\ x_{2} \end{array}\right]+\left[\begin{array}{l}…
Solution to Mathematics for Machine Learning Exercise 7.4
Consider whether the following statements are true or false: (a) The sum of any two convex functions is convex. Solution: True. Let $f_1(\mathbf x)$ and $f_2(\mathbf x)$ be two convex…
Solution to Mathematics for Machine Learning Exercise 7.3
Consider whether the following statements are true or false: (a) The intersection of any two convex sets is convex. Solution: True. Let $\mathcal C_1$ and $\mathcal C_2$ be two convex…
Solution to Mathematics for Machine Learning Exercise 7.1
Consider the univariate function $$f(x)=x^3+6x^2-3x-5.$$Find its stationary points and indicate whether they are maximum, minimum, or saddle points. I will assume you are familiar with Calculus, see the book Calculus…
Solution to Mathematics for Machine Learning Exercise 5.4
Compute the Taylor polynomials $T_n$, $n=0,\dots,5$ of $f(x)=\sin x+\cos x$ at $x_0=0$. Solution: By Definition 5.3, we have to compute $f(x_0)$, $f’(x_0)$, $f^{(2)}(x_0)$, $f^{(3)}(x_0)$, $f^{(4)}(x_0)$, $f^{(5)}(x_0)$. It is not hard…
Solution to Mathematics for Machine Learning Exercise 5.3
Compute the derivative $f’(x)$ of the function $$f(x)=\exp\left(-\frac{1}{2\sigma^2}(x-\mu)^2\right).$$ Solution: Clearly, we have $$\left(-\frac{1}{2\sigma^2}(x-\mu)^2\right)’=-\frac{1}{2\sigma^2}2(x-\mu)=\frac{-(x-\mu)}{\sigma^2}.$$Therefore, by Chain rule (5.32), we have \begin{align*}f’(x)=&\ \exp\left(-\frac{1}{2\sigma^2}(x-\mu)^2\right)\cdot \left(-\frac{1}{2\sigma^2}(x-\mu)^2\right)’\\=&\ \frac{-(x-\mu)}{\sigma^2}\cdot\exp\left(-\frac{1}{2\sigma^2}(x-\mu)^2\right)\end{align*}
Solution to Mathematics for Machine Learning Exercise 5.2
Compute the derivative $f’(x)$ of the logistic sigmoid $$f(x)=\frac{1}{1+\exp(-x)}.$$ Solution: By Chain rule (5.32), we have $$(1+\exp(-x))’=0+\exp(-x)(-x)’=-\exp(-x).$$ By the Quotient rule (5.30), we have\begin{align*}f’(x)=&\ \frac{(1)’(1+\exp(-x))-1(1+\exp(-x))’}{(1+\exp(-x))^2}\\=&\ \frac{0(1+\exp(-x))-(-\exp(-x))}{(1+\exp(-x))^2}\\=&\ \frac{\exp(-x)}{(1+\exp(-x))^2}.\end{align*}
Solution to Mathematics for Machine Learning Exercise 5.1
Compute the derivative $f’(x)$ for $$ f(x)=\log(x^4)\sin(x^3).$$ Solution: By Chain Rule (5.32), we have$$ \big(\sin(x^3)\big)’=\cos(x^3)(x^3)’=3x^2\cos(x^3). $$We also have$$ \big(\log(x^4)\big)’=\big(4\log(x)\big)’=\frac{4}{x}=4x^{-1}. $$Applying Product Rule (5.29), we obtain\begin{align*} f’(x)=&\ \log(x^4) \sin(x^3)\\=&\ \big(\log(x^4)\big)’\sin(x^3)+\log(x^4)\big(\sin(x^3)\big)’\\=&\ 4x^{-1}\sin(x^3)+\log(x^4)3x^2\cos(x^3)\\=&\…