I have started studying vector spaces in depth recently and naturally one would ask a lot of questions, however It seems that all the answers that I'm after require some understanding of matrices and its relevance to vector spaces, so should I go and study matrices first and then go back to vector spaces ?
I should mention that I'm obsessed with understanding things in-depth and dislike things have a lot of computations without theory, so what's the best way to go about learning vector spaces and matrices such that theory and applications are simultaneously present ?
it will hardly be possible to study one without the other. Matrices are, in most cases, just vectors in $K^{n\times n}$ and if you have a linear morphism of finite dimensional $K$-vector spaces $$f:V \rightarrow W$$ you can always express $f$ in terms matrices with respect to chosen bases of $V,W$ respectively. If you know, how $f$ acts on a basis, you know already how $f$ acts on the entire space. You can express this information using matrices.
To get an algebraic understanding for matrices, just start with with some basic book on linear algebra. Michael Artins book on Algebra might be a nice choice. It is always good to start that way, even if you need matrices for computational aspects in discrete mathematics or physics.