In case of a say $n \geq 1$ dimensional system, $\dot{X} =AX$, where $X$ is the $n \times 1$ dimensional vector and $A $ is a $n \times n$ square matrix. The initial condition is specified as $X(0) = X_{0}$.
Now I was thinking about this - If $X(t)$ is a solution on the interval $I$, then how do we prove that the 'double derivative ' of $X$ that is $\frac{d^{2}X}{dt^2}$ is also continuous on $I$?
How do we prove this? only idea coming to mind here is - Differentiability implies Continuity and not the converse. So since $\dot{X}$ exists so it is differentiable, so $\dot{X}$ is continuous, now since $\dot{X}$ is continuous but yet we donot know about the differentiability of $\dot{X}$ that is the double derivative? still we want to prove the continuityof the double derivative of $X$ on $I$?
To elaborate on Alan Muniz's comment:
Suppose you want to solve an ODE of the form $\dot{x}(t)=F(x(t))$, where $x$ should be a function $\mathbb{R}\to\mathbb{R}^n$ and $F\in C^K(\mathbb{R}^n\to\mathbb{R}^n)$. A strong solution has at least one derivative; i.e. $x\in C^1(\mathbb{R}\to\mathbb{R}^n)$. I claim that in fact $x\in C^{K+1}$.
To see this, work by induction. Suppose $x$ has $k$ derivatives, with $k\leq K$. Since the existence of derivatives is preserved by function composition (that's the chain rule), we have $F\circ x\in C^k$. But $\dot{x}=F\circ x$, so $x$ has a $(k+1)$st derivative. Now repeat until $k=K+1$.