I'm going through a book we're using in our intermediate differential equations course, and this is one of the problems that are contained in it. Note that this question is not tagged as homework, because it's not, and I'm actually just trying to solve some of the problems on my own. So I guess I'll jump right into it.
Statement: What is wrong with the following calculation?
$\frac{d}{dt}[exp\int^{t}_{t_0}A(s)ds]=A(t)*[exp\int^{t}_{t_0}A(s)ds],$
so that $exp\int^{t}_{t_0}A(s)ds$ is a fundamental matrix of y' = A(t)y for an arbitrary continuous matrix A(t).
My thoughts: I can't really think of anything other than perhaps that if A(t) is an arbitrary continuous matrix, then $exp[\int^{t}_{t_0}A(s)ds$] need not be invertible, i.e. the solutions (columns) contained within this proposed fundamental matrix need not be linearly independent, thereby contradicting the fundamental matrix assertion. Is that all there is to it? Because I went through the calculation itself, and the equality seems perfectly legit to me.
Further thoughts: Let $J=\int^{t}_{t_0}A(s)ds.$ Then $exp(J)=I+J+\frac{J^{2}}{2!}+...,$ so that by the chain rule:
$\frac{d}{dt}exp(J)=A(t)+JA(t)+\frac{J^{2}}{2!}A(t)+...\neq A(t)+A(t)J+A(t)\frac{J^{2}}{2!}+...=A(t)exp(J),$
unless $A(t)$ and $exp(J)$ commute.
Is this reasoning correct and is this why the calculation fails?
Note that A is a function of s, and therefore need not have the same value for all s. In particular, A(s) and A(s') need not commute. Try redoing your calculation using the limit definition of the derivative and being careful about your assumptions of what commutes. It may be helpful to look up the Zassenhaus formula, which is one way to evaluate the left hand side of your equation, or the Dyson series, which gives the correct form of the solution for your associated differential equation.