Suppose I have an upper triangular $m \times m$ matrix with 1's on the main diagonal and 2's on the first superdiagonal and 0's elsewhere. It is fairly easy to see that this is a full rank matrix with eigenvalues of all 1's.
Now the eigenvalue decomposition is supposed to give me a diagonal matrix of eigenvalues and a matrix of eigenvectors such that
$$ A = Q \Lambda Q^{-1} $$
where the columns of Q are the eigenvectors. Then the inverse of $A$ is
$$ A^{-1} = Q \Lambda^{-1} Q^{-1} = Q \Lambda Q^{-1} = A $$
However, this is not true for the matrix A that I described. So now my question is, why does the eigenvalue decomposition not work for this matrix? If I am not making a mistake somewhere, how then can I invert the matrix A without resorting to something like Gauss Elimination or the sort?
I tried using Matlab's eig function and found that the matrix Q returned does not satisfy $A = Q \Lambda Q^{-1}$ which evaluates to the identity matrix. I try to to rely much on a numerical implementation that involves the inverse of a matrix because I know inverting matrices numerically can be ill-conditioned and lead to severed numerical issues. However, I would expect that an eigenvalue decomposition would yield reasonable results when the eigenvalues are not too small.
Not all matrices are diagonalizable. Yours in particular is not. Suppose it were, so that you can write $A=Q\Lambda Q^{-1}$ with $\Lambda$ diagonal. The eigenvalues of a triangular matrix are its diagonal elements, so we have $\Lambda=I$, but then $Q\Lambda Q^{-1} = QQ^{-1} = I \ne A$.
Why does this go wrong? In the above diagonalization the columns of $Q$ are linearly-independent eigenvectors of $A$. Unfortunately, for your matrix the only eigenvectors are multiples of the standard basis vector $(1,0,0,\dots,0)^T$. Except in the trivial case of $m=1$ there aren’t enough of them to form the diagonalizing matrix $Q$. What you will need to do instead if you want to proceed along these lines is find the Jordan normal form of the matrix.