$\newcommand{\RN}[1]{\text{#1}}$$\newcommand{\tr}{\operatorname{tr}}$An answer specific to $\mathbb{R}^3$ only would be fine.
At around 7:30 of this video (02.08 Tensor Properties III - Continuum Physics - Krishna Garikipati) the coefficients of the characteristic equation are given explicit values in terms of the trace and determinant of a tensor (and of the square of the tensor).
For a second order tensor $A$ of $\mathbb{R}^3$, the principal invariants $\RN{I}, \RN{II},$ and $\RN{III}$ are:
$$\begin{array}{rcl} \RN{I} &=& \tr [A ] \\ \RN{II}& = &\frac{1}{2} (\tr[A])^2 - \frac{1}{2}\tr[A^2] \\ \RN{III} & = & [A] \end{array} $$ which means that the characteristic polynomial of $A$ can be written as: $$\lambda^3 - \RN{I}\ \lambda^2 + \RN{II}\ \lambda - \RN{III}=0 $$
Question: I have never seen this formula before for the coefficients of the characteristic polynomial -- I thought it was always necessary to directly calculate the determinant of $(A-\lambda I)$.
How are these coefficients derived? Are they used often outside of continuum mechanics?
According to Wikipedia, this fact is apparently used frequently in fluid mechanics.
Any hints or advice would be helpful and appreciated.
A diagonalizable 3x3 matrix has three (possibly degenerate) eigenvalues, which are a nicer way to characterize the matrix than individual matrix elements because they don't change if we change the basis we use to represent the vector space the matrix acts on.
The three quantities in your question are other such "invariants" which don't depend on the particular coordinates we choose to label space - they basically just encode the eigenvalues in a different way. $$\mathrm{tr}[A] = \lambda_1 + \lambda_2 + \lambda_3,$$ $$ \mathrm{det}[A] = \lambda_1 \lambda_2 \lambda_3,$$ $$\mathrm{tr}[A^2] = \lambda_1^2 + \lambda_2^2 + \lambda_3^2.$$ These formulae may be readily obtained by diagonalizing the matrix in question.
Since we have three "independent-looking" combinations of the eigenvalues, it seems intuivitely reasonable that taken together they should uniquely specify the eigenvalues i.e. the characteristic polynomial. Specifically: $$(\lambda - \lambda_1 ) ( \lambda - \lambda_2 ) ( \lambda - \lambda_3) = 0$$ $$\lambda^3 - \left(\lambda_1 + \lambda_2 + \lambda_3 \right) \lambda ^2 + (\lambda_1 \lambda_2 + \lambda_1 \lambda_3 + \lambda_2 \lambda_3 ) \lambda -\lambda_1 \lambda_2 \lambda_3 = 0$$ $$\lambda^3 - \mathrm{tr}[A]\lambda ^2 + \frac{1}{2} \left( \mathrm{tr}[A^2] - \mathrm{tr}[A]^2 \right) \lambda - \mathrm{det}[A] = 0,$$ as claimed.