We will define the vector field $X_S$ on $O(n)$ by $X_S (A) = AS$ where $S$ is a skew matrix $(S^T = -S)$ and $A$ is an orthogonal matrix $(A^T = A^{-1})$. Let $S$ and $T$ be skew matrices, how would I find the lie derivative $\mathcal{L}_{X_{S}} X_T$?
I know that the Lie derivative of a vector field along another vector field yields another vector field. However, I do not how to proceed with the Lie derivative of a matrix along a vector field.
Matrix Lie groups have the same property as $\mathbb{R}^n$ that the notions of vector and point get easily mixed up. This can often make computations simpler, but conceptual understanding more difficult.
The Lie algebra $\mathfrak{g}$ of any Lie group $G$ is naturally identified with the tangent space at the identity, so $\mathfrak{g}\simeq T_eG$. The tangent space $T_xG$ at any other point $x\in G$ can also be identified with $\mathfrak{g}$ via left-translation using the group product on $G$: $$L_x:G\to G,\quad L_x(y) = x\cdot y.$$ More precisely, this identification is given the push-forward, i.e. by considering the differential $(dL_x)_e:T_eG\to T_xG$ of the left-translation $L_x$ at the identity $e\in G$.
Returning to the specific matrix group $O(n)$, the Lie algebra is now $\mathfrak{o}(n)$, the space of skew-symmetric matrices. The vector field $X_S$ on the Lie group $O(n)$ is defined by taking a fixed vector $S\in\mathfrak{o}(n)\simeq T_IO(n)$, and left-translating this vector to every point $A\in O(n)$. In other words, $X_S:O(n)\to TO(n)$ is the left-invariant extension of the vector $S\in\mathfrak{o}(n)$.
As a matrix Lie group, the left-translations in $O(n)$ are just the linear maps $$L_A:O(n)\to O(n),\quad L_A(B)=AB,$$ so its differential at the identity becomes essentially the same map $(dL_A)_I(S)=AS$. From a differential geometry point-of-view these are however completely different maps. The latter is a linear map between tangent spaces, whereas the former is a linear map on the group $O(n)$ itself.
What this means is that the vector field $X_S$ takes the conveniently computable explicit form $X_S(A)=AS$. For computational purposes, the values $AS$ are matrices, but conceptually, you should think of these as vectors at the point $A\in O(n)$. That the vector fields $X_S$ really are left-invariant can now be seen by expanding the definitions: $$X_S(L_A(B)) = X_S(AB) = ABS=(L_A)_*BS = (L_A)_*X_S(B).$$
For left-invariant vector fields, the Lie derivative is given by the left-invariant extension of the Lie bracket on the Lie algebra. For matrix Lie algebras, the Lie bracket is given by the commutator $$[S,T] = ST-TS,$$ so the Lie derivative of $X_T$ with respect to $X_S$ is the left-invariant extension of the above. Explicitly, at the point $A\in O(n)$, the vector field $\mathcal{L}_{X_S}X_T$ gives the vector $$(\mathcal{L}_{X_S}X_T)_A = [X_S,X_T]_A = (L_A)_*[S,T] = A(ST-TS).$$