Prove that the zero solution is unstable for the system $x' = A(t)x$ with $A = \begin{pmatrix} \frac{1}{2} - \cos(t) & 12 \\ 147 & \frac{3}{2} + \sin(t)\end{pmatrix}$.
I've tried the following:
The system is equivalent to $x' = Bx + D(t)x$, with $B= \begin{pmatrix} \frac{1}{2} & 12 \\ 147 & \frac{3}{2} \end{pmatrix}$ and $D(t) = \text{diag}(-\cos(t), \sin(t))$, then find the eigenvalues of $B$ and notice that one of them is positive. Therefore the linear system $x' = Bx$ has unstable zero solution.
I'm unsure however on how to translate that into instability of the original system, since the system is not autonomous. The question is in the Floquet theory section of the book so presumably there is a way to find/involve the characteristic multipliers/exponents. Any help is appreciated.
Found an answer involving the characteristic multipliers.
Let $\phi(t)$ be the principal fundamental matrix solution at $t=0$, (i.e. $\phi(0) = I$), then, from Floquet's theorem there exists a matrix B (possibly complex) such that: $\phi(T) = e^{TB}$
Let $\lambda_1, \lambda_2$ be the two (potentially equal) eigenvalues of $e^{TB}$, i.e., $\lambda_1, \lambda_2$ are the characteristic multipliers of the system. Then $\lambda_1\lambda_2 = det(e^{TB}) = det(\phi(T))$. Applying Liouville's formula:
$det(\phi(T)) = det(\phi(0))e^{\int_0^T tr(A(t))dt } = e^{\int_0^T tr(A(t))dt }$ (since $\phi(0) = I$)
$A(t)$ is $2\pi$ periodic and $tr(A) = 2 - \cos(t) + \sin(t)$, and since $\int_0^{2\pi} \sin(t) = \int_0^{2\pi} \cos(t) = 0$ we have that:
$\int_0^T A(t) dt = \int_0^{2\pi} (2 - \cos(t) + \sin(t)) dt = \int_0^{2\pi} 2dt = 4\pi$
so $\lambda_1\lambda_2 = e^{4\pi}$. Then the product of the modulus $|\lambda_1||\lambda_2| = |\lambda_1\lambda_2| > 1$. This means that at least one of $|\lambda_1|, |\lambda_2|$ is larger than $1$.
Then by one of the most important theorems in Floquet theory (Theorem 2.89, Chicone (2006)), the zero solution is unstable.