I understand matrices in the light of linear transformations, and I know that the inverse of a matrix ($A^{-1}$) applied after the transformation $A$ essentially gets you back to where you started — in other words, effectively "cancelling" the transformation.
However, the procedure for finding the inverse of a $3\times3$ matrix seems completely arbitrary. The fact that we find the matrix of minors, apply positive and negative signs in a particular way to get a co-factor matrix and so on simply does not translate for me intuitively.
It doesn't make sense why because it does not tie in with the geometric approach I've been using to learn matrices. How do I make sense of how the inverse of a $3 \times 3$ matrix is computed?
The reason we use this as a formula, is because of the fact that for a matrix $A$, \begin{align*} A\cdot \text{adj}(A)=\det{(A)} I \end{align*} where adj($A$) is the adjugate matrix obtained by transposing the cofactor matrix. The theory behind this is not very difficult, but it is a lot to put in one answer without a clear starting point. You can find a proof of the above identity here, and if you would rather a book to walk you through the process, I would recommend C.W. Curtis - Linear Algebra: an introductory approach. Springer 1984