Transforming a vector/matrix fraction to its product

Question

Ok, this might be a really simple one, but I just don't know where these kind of algebra rules are defined..

Why is $\frac{x^2}{A} = x^TA^{-1}x$ if you transform it from normal algebra to a vector notation? Here $x$ is a vector and $A$ a matrix.

Thanks!

Answer 1

Unlike the commenters, I understand your question, but it is based on a mistaken idea that we are somehow just transforming algebra on the reals or complex numbers into linear algebra equations. This is not the case. Linear algebra is built up from its own axioms, based on the needs of the problem. Because the axioms of Linear algebra are very similar to the axioms of the real numbers, the expressions we get are also similar, such as finding $$x^TA^{-1}x$$ used in a way that we are used to seeing $$\dfrac {x^2}A$$ used in real or complex algebra.

But we didn't get $x^TA^{-1}x$ by somehow transforming $\dfrac {x^2}A$. We get it by building up logically. How? That I cannot answer because I don't know where you are encountering this expression, so I cannot say why it appears. Depending on the problem at hand, it may come up in a number of different ways.

But there is two general rules that do help here:

1. Matrix multiplication is not commutative. $AB \ne BA$ in general, even when it is defined. I.e., when $A$ and $B$ are square matrices of the same dimension. This is why you never see division involved in linear algebra, except when by scalars. As you know, division is multiplying by the inverse, but the expression $$\frac AB$$ is ambiguous, even if $B$ is invertible. Since $B^{-1}A \ne AB^{-1}$, which one should be used for $\frac AB$? So we don't use the expression $\frac AB$. Instead we always use either $AB^{-1}$ or $B^{-1}A$. Which we use depends on whether we needed to multiply $B^{-1}$ on the left or on the right in our calculation.
2. You can only multiply things together when the multiplication is defined: if $A$ is an $m \times n$ matrix and $B$ is a $p \times q$ matrix, then $AB$ is only defined if $n = p$. In particular, in an $n$-dimensional space, a vector $x$ can be considered as an $n \times 1$ column matrix, and $x^T$ is therefore a $1 \times n$ row matrix. $A^{-1}$ is an $n \times n$ square matrix.

As a result, if $n > 1$, you can only multiply $A^{-1}$ and two copies of $x$ together in a few ways:

• $Ax$ is $n \times n$ times $n \times 1$ which is defined. It gives you another $n \times 1$ matrix. Also $x^TA$ is defined and gives you another $1 \times n$ matrix.
• Neither $xA$ nor $Ax^T$ are defined.
• $x^Tx$ is $1 \times n$ times $n \times 1$ and gives a $1 \times 1$ matrix, (which can be treated as a scalar).
• $xx^T$ is $n \times 1$ times $1 \times n$ and gives an $n \times n$ matrix.
• Neither $xx$ nor $x^Tx^T$ are defined.

When you combine these with the third factor, the only defined products are:

• $x^TA^{-1}x$ is a $1\times 1$ matrix (which again can be treated as a scalar).
• $A^{-1}xx^T$ and $xx^TA^{-1}$ are $n \times n$ matrices.
• You can also "cheat" and get $x^TxA^{-1}$. The cheat here is that we are considering $x^Tx$ as a scalar and are using scalar multiplication, not matrix multiplication. As a scalar multiple of $A^{-1}$, the result is an $n \times n$ matrix.

Of these three forms, the first is by far the most common one encountered. It can be thought of as the magnitude of $x$, weighted by the matrix $A^{-1}$.