Solution to Abstract Algebra by Dummit & Foote 3rd edition Chapter 1.6 Exercise 1.6.20
Let $G$ be a group and let $\mathsf{Aut}(G)$ be the set of all isomorphisms $G \rightarrow G$. Prove that $\mathsf{Aut}(G)$ is a group under function composition.
Solution: We need to verify that the three group axioms are satisfied: associativity, identity, and inverses.
(1) We know from set theory that function composition is always associative.
(2) Note that $\mathsf{id}_G$ is a bijection and trivially a homomorphism, so that $\mathsf{id}_G \in \mathsf{Aut}(G)$. Finally, we have $$\mathsf{id}_G \circ \varphi = \varphi \circ \mathsf{id}_G = \varphi$$ for all isomorphisms $\varphi : G \rightarrow G$, so that $\mathsf{id}_G$ is an identity element under composition.
(3) Given $\varphi \in \mathsf{Aut}(G)$, we know from set theory that an inverse $\varphi^{-1}$ exists. This inverse is a homomorphism, as we show. If $a,b \in G$, then $$\varphi(\varphi^{-1}(ab)) = ab = \varphi(\varphi^{-1}(a))\varphi(\varphi^{-1}(b)) = \varphi(\varphi^{-1}(a) \varphi^{-1}(b)).$$ Since $\varphi$ is injective, we have $$\varphi^{-1}(ab) = \varphi^{-1}(a) \varphi^{-1}(b).$$ Thus $\varphi^{-1}$ is a homomorphism, and we have $$\varphi \circ \varphi^{-1} = \varphi^{-1} \circ \varphi = \mathsf{id}_G.$$ Thus $\mathsf{Aut}(G)$ is a group under function composition.
