Good Textbook on Matrices

Question

I have just started studying Optimization, and I have realized that I am severely lacking some background material in Linear Algebra. I know about as much as a (weak) first counrse in undergraduate linear algebra. In particular, I know more about Linear Algebra more in terms of abstract vector spaces, rather than matrices. (since I have also studied it in an Abstract Algebra class)

Could you please recommend me a book that would cover the propreties of matrices useful in in applied mathematics/multivariable analysis?

What I know about Linear Algebra:

• Basic algebra with matrices (add, multiply)

• Gaussian elimination

• Vector spaces, subspaces, quotient spaces. I know every linear transformation from a finite-dimensional vector space has a representation as a matrix $\in F^{n \times m}$

• Rank-Nullity Theorem

• Definition of eigenvalues, eigenvectors, and characteristic polynomials

• I know a little bit about the Jordan Canonical Form and Rational Canonical Form

What I Would Like to Learn:

• Basic matrix information which is useful in elementary applied math/elementary multivariable analysis. For example, the basics properties of positive definite matrices, quadratic forms, their relationships with eigenvalues, etc.

• Various factorizations useful in applied math, such as SVD

• Basic techniques/manipulations/tricks with matrix algebra

Answer 1

Like I said in my comment above, it's a little unclear in what capacity you're familiar with "abstract" linear algebra, but to gain a good clear understanding of matrix operations I'd look at the following books.

Basic Introductions

These are books that I would consider beginner-level texts. I'm assuming you're looking for something harder than these, but I'll include them for completeness.

• Linear Algebra and its Applications by Gilbert Strang is a very popular text. This takes linear algebra from mostly an engineering and applications perspective.
• Linear Algebra Done Right by Sheldon Axler is a great abstract/proof-based approach to the subject. This is a great book for learning basic proof methods in linear algebra and it contains within it the seeds of functional analysis.
• The No Bullshit Guide to Linear Algebra by Ivan Savov is a great compendium of everything may need to know about lin alg for undergraduate applications. I particularly like the organization of the book, but it may be sparse in certain respects.

Matrix Analysis

More advanced books that deal with matrix analysis which could be used for calculus of matrices, more advanced decompositions, and also bridging the way into functional analysis/operator theory.

• Matrix Analysis by Horn and Johnson contains almost encyclopedic knowledge of matrix decompositions.
• Matrix Analysis by Rajendra Bhatia is similarly a good book along these lines as well.
• Linear Algebra and its Applications by Peter Lax is a good advanced book that has a mix of theory and applications.