Solution to Abstract Algebra by Dummit & Foote 3rd edition Chapter 4.2 Exercise 4.2.9
Solution: Let $G$ be a group of order $p^k$ and $H \leq G$ a subgroup with $[G : H] = p$. Now $G$ acts on the conjugates $gHg^{-1}$ by conjugation, since $$g_1g_2 \cdot H = (g_1g_2)H(g_1g_2)^{-1} = g_1(g_2Hg_2^{-1})g_1^{-1} = g_1 \cdot (g_2 \cdot H)$$ and $1 \cdot H = 1H1 = H$. Moreover, under this action we have $H \leq \mathsf{stab}(H)$. By Exercise 3.2.11, we have $$[G : \mathsf{stab}(H)][\mathsf{stab}(H):H] = [G:H] = p,$$ a prime.
If $[G : \mathsf{stab}(H)] = p$, then $[\mathsf{stab}(H):H] = 1$ and we have $H = \mathsf{stab}(H)$; moreover, $H$ has exactly $p$ conjugates in $G$. Let $\varphi : G \rightarrow S_p$ be the permutation representation induced by the action of $G$ on the conjugates of $H$, and let $K$ be the kernel of this representation. Now $K \leq \mathsf{stab}(H) = H$. By the first isomorphism theorem, the induced map $\overline{\varphi} : G/K \rightarrow S_p$ is injective, so that $|G/K|$ divides $p!$. Note, however, that $|G/K|$ is a power of $p$ and that the only powers of $p$ that divide $p!$ are 1 and $p$. So $[G:K]$ is 1 or $p$. If $[G:K] = 1$, then $G = K$ so that $gHg^{-1} = H$ for all $g \in G$; then $\mathsf{stab}(H) = G$ and we have $[G:\mathsf{stab}(H)] = 1$, a contradiction. Now suppose $[G:K] = p$. Again by Exercise 3.2.11 we have $[G:K] = [G:H][H:K]$, so that $[H:K] = 1$, hence $H = K$. Again, this implies that $H$ is normal so that $gHg^{-1} = H$ for all $g \in G$, and we have $[G:\mathsf{stab}(H)] = 1$, a contradiction. Thus $[G:\mathsf{stab}(H)] \neq p$.
If $[G : \mathsf{stab}(H)] = 1$, then $G = \mathsf{stab}(H)$. That is, $gHg^{-1} = H$ for all $g \in G$; thus $H \leq G$ is normal.
Now let $G$ be a group of order $p^2$. By Cauchy’s Theorem, $G$ contains an element $x$ of order $p$. Then $[G:\langle x \rangle] = p$, hence $\langle x \rangle$ is normal.