This is probably a very easy question, I'm just not strong at the concept.
Let $ A $ be an $ n \times n $ matrix with coefficient in a field $ F $. Then $ A $ determines a linear map $ T: F^n \to F^n $ and gives $ F^n $ a $ F[x] $-module. The characteristic polynomial of $ A $ is $ \text{det}(xI - A) $, while the minimal polynomial is defined to be the monic polynomial generating $ \text{Ann}(F^n) $ as the above $ F[x] $-module.
When finding the invariant factors, the method I see is to calculate the minimal polynomial by using the determinant formula. For example, say it is $ (x-2)^2(x-3) $. Now the minimal polynomial is either $ (x-2)(x-3) $ or $ (x-2)^2(x-3) $. But in the next step, the book substitutes in $ (A- 2I)(A-3I) $ and check if it's zero. Why is this possible? This looks like the Cayley-Hamilton theorem, but isn't that only for characteristic polynomials?
This is the most convenient way. The minimal polynomial is the monic polynomial of minimum degree (on the given field) which vanishes at the endomorphism ($A$ in your example). Certainly the polynomial you found (the characterisic one) vanishes at $A$ but you don't know if it is also the minimal polynomial, so you must check if its proper divisors (with all irreducible factors) vanish at the matrix.