The exponential of the matrix $At$ is related to linear systems of ordinary differential equations. It is given by the concise yet apparently somewhat arbitrary formula $e^{At} = Y(t)Y(0)^{-1}$, where the order of operations between inverting the second $Y$ and evaluating it at $0$ doesn't matter.
Oftentimes in linear algebra, there are multiple ways to view basic operations (column and row approaches to multiplication), or multiple ways operations could be defined, with one arbitrary choice being declared correct. Do expressions similar to the actual $e^{At}$ formula such as $$Y(0)Y(t)^{-1}$$$$Y(t)^{-1}Y(0)$$$$Y(0)^{-1}Y(t)$$ relate to the ODE system as well, or is there something special about the actual formula that conveys informations about the system?
I do not think you define the exponential of a matrix via systems of equations (although that is possible of course). You define it independently of any ODE as a series $$ e^A=I+A+\frac{A^2}{2!}+\dots $$ which is convergent for any matrix $A$. Then you prove some basic properties of this function and, in particular, you prove that $$ \frac{d}{dt}e^{At}=Ae^{At} $$ a property that is telling you that $e^{At}x_0$ is a solution of the differential system $x'=Ax$, and all you need to adjust is the initial value, so you need $x_0=x(0)$. In other words, $$ x(t)=e^{At}x(0), $$ or, for any system of solutions $Y(t)$ (in particular for the fundamental system), $$ Y(t)=e^{At}Y(0) $$ from which your relation follows. But I do not think the distinction between representations as $Y(t)Y^{-1}(0)$ and similar ones is relevant.