Cannot find this question, though I find it hard to believe that it hasn't been answered somewhere. All I really have here are the requirements for something to be considered a matrix norm, and that my A must be non-singular.
I got that $||A||\cdot ||A^{-1}|| \le ||AA^{-1}|| = ||1||$ from the triangle inequality, so if I could drop that pesky $\le$ operator and replace it with proper equality, I would know that $||A^{-1}||$ was in fact the inverse of $||A||$, a.k.a.$||A^{-1}|| = ||A||^{-1}$. But I can't think of how to show equality, or if I even can.
I also may be making bad assumptions in that $||1|| = 1$, because the identity matrix may not necessarily map to 1? Though I can't see why it wouldn't. My linear is very shaky, hopefully I'm going in the right direction.
It is rarely true that the norm of the inverse is the inverse of the norm. E.g., for $A$ the diagonal two-by-two diagonal matrix with diagonal entries $1$ and $2$, the norm is $2$. The inverse has diagonal entries $1$ and $1/2$, and has norm $1$.
EDIT: here $\|A\|=\sup_{|x|\le 1} |Ax|$, the usual operator norm.