Chapter 5 Differentiation
Exercise 21
(By analambanomenos) I’m going to show this for the infinitely differentiable case (and even that won’t be complete). Cauchy’s function $F(x)=e^{-1/x^2}$ for $x\neq 0$, $F(0)=0$, is the classic counterexample of a non-constant, infinitely differentiable real function such that $f^{(n)}(0)=0$ for all orders $n$ (you can easily find a proof of this, or you can try it yourself). We can use this to define a function $\tilde{F}(x)=F(x)$ for $x>0$, $\tilde{F}(x)=0$ for $x\le0$, which is infinitely differentiable everywhere on $\mathbf R$ and whose zero set is $(-\infty,0]$. Finally, for $a<b$ let $$F_{a,b}(x)=\tilde{F}(x-a)\tilde{F}(b-x),$$ an infinitely differentiable function on $\mathbf R$ whose zero set is $(-\infty,a]\cup[b,\infty)$.
The complement of a closed set $E$ of $\mathbf R$ is an open set consisting of a (possibly infinite) collection of open intervals $(a_n,b_n)$. The function $f(x)=\sum F_{a_n,b_n}(x)$ is well-defined since at most one of the terms in the sum is non-zero for any given $x$, infinitely differentiable everywhere on $\mathbf R$, and has zero set $E$.
Exercise 22
(By analambanomenos)
(a) Suppose $f(a)=a$ and $f(b)=b$ for $a<b$. By Theorem 5.10, there is a point $t$, $a<t<b$, such that $f’(t)=\bigl(f(b)-f(a)\bigr)/(b-a)=1$, contradicting $f’(t)\neq 1$ for all real $t$.
(b) If $t=f(t)=t+(1+e^t)^{-1}$, then $(1+e^t)^{-1}=0$, which is impossible. We have $$f(t)=1-\frac{e^t}{(1+e^t)^2}\quad\quad f”(t)=\frac{e^t(e^t-1)}{(1+e^t)^3}.$$ Since $f”(t)$ has a single zero, at $t=0$, $f’(t)$ decreases from $\lim_{t\rightarrow-\infty}f’(t)=1$ to $f’(0)=3/4$, then increases to $\lim_{t\rightarrow\infty}f’(t)=1$, so that the range of $f’$ is $[3/4,1)$.
(c) Since $x_{n+1}=f(x_n)$ and $x_{n}=f(x_{n-1})$, by Theorem 5.10 there is a point $t$ between $x_{n-1}$ and $x_n$ such that $$(x_{n+1}-x_n)=(x_n-x_{n-1})f’(t).$$ Hence
$$|x_{n+1}-x_n|<A|x_n-x_{n-1}|<A^2|x_{n-1}-x_{n-2}|<\cdots<A^{n-1}|x_2-x_1|.$$ So for $m$ and $n$ greater than $N$, we have
\begin{align*}
|x_n-x_m| &\le |x_n-x_{n+1}|+\cdots+|x_{m-1}-x_m| \\
&\le (A^{n-1}+\cdots+A^{m-2})|x_2-x_1| \\
&\le |x_2-x_1|\sum_{k=N}^{\infty}A^k \\
&\le \frac{|x_2-x_1|}{1-A}A^N
\end{align*} Since $A<1$, this last term goes to 0 as $N\rightarrow\infty$, so $\{x_n\}$ is a Cauchy sequence, converging to $x$. Since $f$ is differentiable, it is continuous, hence
$$f(x)=\lim_n f(x_n)=\lim_n x_{n+1}=x$$ that is, $x$ is a fixed point of $f$.
(d) This zig-zag path is described in $\mathbf R^2$ with respect to the graph of $f$ in that space. It starts with the point $\bigl(x_1,x_2=f(x_1)\bigr)$ on the graph of $f$, goes horizonatally until it meets the diagonal $y=x$ at $(x_2,x_2)$ then goes vertically until it hits the graph of $f$ again at $\bigl(x_2,x_3=f(x_2)\bigr)$, and so forth. It will zig-zag or spiral to the point on the graph of $f$ corresponding to a fixed point of $f$, where it crosses the diagonal $y=x$.
Exercise 23
(By analambanomenos) The fixed points of $f$ are the zeros of $g(x)=f(x)-x$, of which there are at most three. Since $g(-2)=-1/3$, $g(-1)=1$, $g(0)=1/3$, $g(1)=-1/3$, and $g(2)=1$, there are three fixed points of $f$, lying in the intervals $(-2,-1)$, $(0,1)$, and $(1,2)$, as asserted.
Note that $\Delta x_n=x_n-x_{n-1}=f(x_{n-1})-x_{n-1}=g(x_{n-1})$. For $x<\alpha$, $g(x)<0$, and $g$ decreases monotonically to $-\infty$ as $x\rightarrow-\infty$. Hence if $x_1<\alpha$, then $g(x_{n-1})=\Delta x_n<0$, so $x_n<x_{n-1}$ and $\Delta x_{n+1}=g(x_n)<\Delta x_n$. Hence $x_n\rightarrow-\infty$ as $n\rightarrow\infty$.
Similarly, for $x>\gamma$, $g(x)>0$, and $g$ increases monotonically to $+\infty$ as $x\rightarrow+\infty$. Hence if $x_1>\gamma$, then $g(x_{n-1})=\Delta x_n>0$, so $x_n>x_{n-1}$ and $\Delta x_{n+1}=g(x_n)>\Delta x_n$. Hence $x_n\rightarrow+\infty$ as $n\rightarrow\infty$.
For $x\in(\alpha,\beta)$ we have $g(x)>0$, and for $x\in(\beta,\gamma)$ we have $g(x)<0$. I want to compare $g$ with the linear function $\beta-x$, which also has a zero at $x=\beta$ and has a slope of $-1$. Since $g’(\beta)=\beta^2-1>-1$, we have $g(x)<\beta-x$ for $x\in(\alpha,\beta)$ and $g(x)>\beta-x$ for $x\in(\beta,\gamma)$.
Putting this together, we get that if $x_n\in(\alpha,\beta)$, then $0<\Delta x_{n+1}<\beta-x_n$ so that $x_{n+1}\in(x_n,\beta)$. Similarly, if $x_n\in(\beta,\gamma)$, then $\beta-x<\Delta x_{n+1}<0$ so that $x_{n+1}\in(\beta,x_n)$.
That is, if $x_1\in(\alpha,\beta)$, then $\{x_n\}$ is a monotonically increasing sequence with upper bound $\beta$, and if $x_1\in(\beta,\gamma)$, then $\{x_n\}$ is a monotonically decreasing sequence with lower bound $\beta$. In either case they must have a limit $\beta’$. This must be a zero of $g$ (and so a fixed point of $f$) since otherwise $g(\beta’)\neq 0$, so for $x_n$ close to $\beta’$, the value of $|\Delta x_n|$ would be larger than $|\beta’-x_n|$ which would contradict $\beta’$ being a limit point of the monotonic sequence $\{x_n\}$. Hence $\beta’=\beta$.
Exercise 24
(By analambanomenos) The distance between the successive elements of the sequence $x_n=f(x_{n-1})$ is given by the function $F(x)=f(x)-x$, while the distance between the successive elements of the sequence $y_n=g(y_{n-1})$ is given by the function $G(y)=g(y)-y$. We have
$$\displaylines{
F(x)=-\frac{1}{2}\biggl(x-\frac{\alpha}{x}\biggr) \cr
F\bigl(\sqrt{\alpha})=0,\quad F’\bigl(\sqrt{\alpha}\bigr)=-1 \cr
G(y)=\frac{\alpha-y^2}{1+\alpha}\cr
G\bigl(\sqrt{\alpha})=0,\quad G’\bigl(\sqrt{\alpha}\bigr)=-\frac{2}{1+\sqrt{\alpha}}.
}$$ By Taylor’s theorem we have near $\sqrt{\alpha}$
\begin{align*}
F(x) &= -(x-\sqrt{\alpha})+K_1(x-\sqrt{\alpha})^2 \\
G(y) &= -\frac{2}{1+\sqrt{\alpha}}(y-\sqrt{\alpha})+K_2(y-\sqrt{\alpha})^2
\end{align*}Now $\sqrt{\alpha}-x$ is the distance from $x$ to $\sqrt{\alpha}$, so we see that as $x_n$ approaches $\sqrt{\alpha}$ the differences between the successive elements of the first sequence become very close to the distance to $\sqrt{\alpha}$, but not so much in the case of the second sequence, which has the additional factor of $2/\bigl(1+\sqrt{\alpha})$. In other words, the difference between the successive steps of the $x_n$ and the distance to $\sqrt{\alpha}$ is quadratic in $x-\sqrt{\alpha}$, while in the case of the second sequece the difference is only linear in $y-\sqrt{\alpha}$.
Exercise 25
(By analambanomenos)
(a) The tangent to the graph of $f$ at the point $\bigl(x_n,f(x_n)\bigr)$ has the equation $$y-f(x_n)=f’(x_n)(x-x_n).$$ Letting $y=0$ and solving for $x$, we see that this line intersects the $x$-axis at the point $(x_{n+1},0)$.
(b) Note that if $x_n>\xi$ then $f(x_n)>0$ so that $x_{n+1}=x_n-f(x_n)/f’(x_n)<x_n$. By Theorem 5.10, there is a $t\in(\xi,x_n)$ such that $$f(x_n)=f(x_n)-f(\xi)=f’(t)(x_n-\xi)\quad\hbox{or}\quad \xi=x_n-\frac{f(x_n)}{f’(t)}\le x_n-\frac{f(x_n)}{f’(x_n)}=x_{n+1}$$ since $f”\ge0$ implies $f’(t)\le f’(x_n)$. If for some $m$ $x_{m}=\xi$ then $x_{m+1}=x_{m+2}=\cdots=\xi$ and so the infinite sequence converges to $\xi$. Otherwise, it is a monotonically decreasing sequence with lower bound $\xi$ and so by Theorem 3.14 converges to a limit $y\ge\xi$.
To see that $y=\xi$, let $g$ be the continuous function defined in part (e), $g(x)=x-f(x)/f’(x)$. Assume that $y>\xi$. Then $f(y)>0$, so $g(y)<y$. Let $\varepsilon>0$ be small enough so that $g(y)+\varepsilon<y$. Then there is a $\delta>0$ such that $\big|g(x)-g(y)\big|<\varepsilon$ if $|x-y|<\delta$. Let $N$ be large enough so that $|x_N-y|<\delta$. Then $g(x_N)<g(y)+\varepsilon<y$, but this contradicts $g(x_N)=x_{N+1}\ge y$. Hence $y=\xi$.
(c) Letting $\alpha=x_n$ and $\beta=\xi$ in Theorem 5.15, and using $f(x_n)=f’(x_n)(x_n-x_{n+1})$, there is some $t_n\in(\xi,x_n)$ such that
\begin{align*}
f(\xi) &= f(x_n)+f’(x_n)(\xi-x_n)+\frac{f”(t_n)}{2}(\xi-x_n)^2 \\
0 &= f’(x_n)(x_n-x_{n+1})+f’(x_n)(\xi-x_n)+\frac{f”(t_n)}{2}(\xi-x_n)^2 \\
0 &= f’(x_n)(\xi-x_{n+1})+\frac{f”(t_n)}{2}(\xi-x_n)^2 \\
x_{n+1}-\xi &= \frac{f”(t_n)}{2f’(x_n)}(x_n-\xi)^2
\end{align*}(d) Part (c) gives us
\begin{align*}
(x_{n+1}-\xi) &\le A(x_n-\xi)^2 \\
&\le A\cdot A^2(x_{n-1}-\xi)^{2^2} \\
&\le A\cdot A^2\cdot A^{2^2}(x_{n-2}-\xi)^{2^3} \\
&\le\cdots \\
&\le A^{1+2+2^2+\cdots+2^{n-1}}(x_1-\xi)^{2^n} = A^{2^n-1}(x_1-\xi)^{2^n} = \frac{1}{A}\bigl(A(x_1-\xi)\bigr)^{2^n}.
\end{align*}The algorithm described in Exercises 3.16 and 3.18 is Newton’s method applied to the functions $x^2-\alpha$ and $x^p-\alpha$, respectively.
(e) We have $g(\xi)=\xi$ if and only if $f(\xi)=0$. Since $$g’(x)=1-\frac{f’(x)^2-f(x)f”(x)}{f’(x)^2}=\frac{f(x)f”(x)}{f’(x)^2}\le f(x)\frac{M}{\delta^2},$$ $g’(x)$ tends to 0 as $x$ approaches $\xi$.
(f) For $f(x)=x^{1/3}$, note that $$x_{n+1}=x_1-\frac{x_n^{1/3}}{(1/3)x_n^{-2/3}}=-2x_n$$ so that $x_n=(-2)^{n-1}x_1$. That is, $\{x_n\}$ is an alternating sequence such that $|x_n|\rightarrow\infty$ as $x\rightarrow\infty$.
Exercise 26
(By analambanomenos) I’m going to show this for vector-valued functions $\mathbf f=(f_1,\ldots,f_k)$ mapping $[a,b]$ into $\mathbf R^k$ such that $\mathbf f(a)=\mathbf 0$ and such that $\big|\mathbf f’(x)\big|\le A|\mathbf f(x)|$, since this is needed for Exercise 28.
If $A=0$, then $\mathbf f’=0$, so $\mathbf f=\mathbf f(a)=\mathbf 0$ by Theorem 5.11(b). So assume that $A>0$. Following the hint, let $\delta>0$ be small enough so that $\delta<1/A$ and $x_0=a+\delta\le b$. For $i=1,\ldots,k$, let $$M_{i,0}=\sup\big|f_i(x)\big|, \quad M_{i,1}=\sup\big|f_i’(x)\big|$$ for $a\le x\le x_0$. By Theorem 5.19, for any $x\in(a,x_0)$ and for every $i=1,\ldots,k$ there is a $t_i\in(a,x)$ such that $$\big|f_i(x)\big|=\big|f_i(x)-f_i(a)\big|=\big|f_i’(t)\big|(x-a)\le M_{i,1}(x-a)\le AM_{i,0}\delta.$$ Since $A\delta<1$, this can only happen if $M_{i,0}=0$, so $f_i(x)=0$ in $[a,a+\delta]$ for each $i=1,\ldots,k$, that is, $\mathbf f=\mathbf 0$ in $[a,a+\delta]$. Repeating this argument with $a+\delta$ replacing $a$, we get $\mathbf f(x)=\mathbf 0$ in $[a,a+2\delta]$. After about $(b-a)/\delta$ steps, we’ve shown that $\mathbf f(x)=\mathbf 0$ in $[a,b]$.
Exercise 27
(By analambanomenos) Following the hint, let $f_1$, $f_2$ be two solutions of the initial value problem and let $f=f_1-f_2$. Then $f$ is differentiable on $[a,b]$, $f(a)=f_1(a)-f_2(a)=0$, and
$$\big|f’(x)\big|=\big|f_1′(x)-f_2′(x)\big|=\big|\phi(x,f_1(x))-\phi(x,f_2(x))\big|\le A\big|f_1(x)-f_2(x)\big|=A\big|f(x)\big|.$$ Applying Exercise 27, we get $f(x)=0$ for all $x\in[a,b]$.
Exercise 28
(By analambanomenos) If there is a constant $A$ such that $$\big|\boldsymbol\phi(x,\mathbf y_2)-\boldsymbol\phi(x,\mathbf y_1)\big|\le A|\mathbf y_2-\mathbf y_1|,$$ then $\mathbf y’=
\boldsymbol\phi(x,\mathbf y)$, $\mathbf y(a)=\mathbf c$ has at most one solution. For suppose $\mathbf f_1$ and $\mathbf f_2$ are two such solutions, and let $\mathbf f=
\mathbf f_1-\mathbf f_2$. Then $\mathbf f$ is differentiable on $[a,b]$, $\mathbf f(a)=\mathbf 0$, and $$\big|\mathbf f’(x)\big|= \big|\boldsymbol\phi(x,\mathbf f_1(x))-
\boldsymbol\phi(x,\mathbf f_2(x))\big|\le A|\mathbf f_1-\mathbf f_2|=A|\mathbf f|.$$ Hence by the vector-valued version of Exercise 26 shown above, $\mathbf f(x)=\mathbf 0$ for all $x\in[a,b]$.
Exercise 29
(By analambanomenos) Using the notation of Exercise 28, $\boldsymbol\phi(x,\mathbf y)=\bigl(\phi_1(x,\mathbf y),\ldots,\phi_k(x,\mathbf y)\bigr)$ where
$$\mathbf \phi_j(x,\mathbf y)=
\begin{cases}
y_{j+1} & j=1,\ldots,k-1 \\
f(x)+\sum_{i=1}^k g_i(x)y_i = f(x)+\mathbf g(x)\cdot\mathbf y & j=k
\end{cases}$$ where $\mathbf g(x)=\bigl(g_1(x),\ldots,g_k(x)\bigr)$. Then
\begin{align*}
\big|\boldsymbol\phi(x,\mathbf y_2)-\boldsymbol\phi(x,\mathbf y_2)\big|^2 &= \sum_{j=1}^k\big|\phi_j(x,\mathbf y_2)-\phi_j(x,\mathbf y_1)\big|^2 \\
&= \sum_{j=1}^{k-1}|y_{2,j+1}-y_{1,j+1}|^2 + \big|\mathbf g(x)\cdot(\mathbf y_2-\mathbf y_1)\big|^2 \\
&\le |\mathbf y_2-\mathbf y_1|^2+|\mathbf g(x)|^2|\mathbf y_2-\mathbf y_1|^2 \\
&=\bigl(1+|\mathbf g(x)|^2\bigr)|\mathbf y_2-\mathbf y_1|^2 \\ \\
\big|\boldsymbol\phi(x,\mathbf y_2)-\boldsymbol\phi(x,\mathbf y_2)\big| &\le \sqrt{\bigl(1+|\mathbf g(x)|^2\bigr)}\;|\mathbf y_2-\mathbf y_1|
\end{align*}Since the $g_i(x)$ are continuous on $[a,b]$, they are bounded, so that there is a constant $A$ such that $\sqrt{\bigl(1+|\mathbf g(x)|^2\bigr)}\le A$ for all $x\in[a,b]$. Hence by Exercise 28, the equation has at most one solution.
