How to combine $(x - \mu_0)^TA(x - \mu_0) - (x - \mu_1)^T A (x - \mu_1)$

56 Views Asked by At

How to combine

$(x - \mu_0)^TA(x - \mu_0) - (x - \mu_1)^T A (x - \mu_1)$

where

  • $x$, $\mu_0$, and $\mu_1$ are all vectors of shape $m \times 1$, and
  • $A$ is a matrix of shape $m \times m$

I was expecting something with $x^Tx$ cancelled out in the result, but I haven't figured it out, yet.

Suppose $A$ is invertible, I am also interested in

$(x - \mu_0)^T A^{-1} (x - \mu_0) - (x - \mu_1)^T A^{-1} (x - \mu_1)$

I thought whatever the above result is, the deduction should apply to the $A^{-1}$ case, as well.

1

There are 1 best solutions below

6
On BEST ANSWER

$$(x - \mu_0)^TA(x - \mu_0) - (x - \mu_1)^T A (x - \mu_1) $$ is equal to $$x^TAx - x^TA \mu_0 - \mu_0^T A x + \mu_0^T A \mu_0-(x^TAx - x^TA \mu_1 - \mu_1^T A x + \mu_1^T A \mu_1)$$ Notice that $x^T A x$ cancel out $$- x^TA \mu_0 - \mu_0^T A x + \mu_0^T A \mu_0 + x^TA \mu_1 + \mu_1^T A x - \mu_1^T A \mu_1$$ You can combine similar terms like $$ \mu_0^T A \mu_0 - \mu_1^T A \mu_1 + x^TA (\mu_1-\mu_0) + (\mu_1-\mu_0)^T A x $$ If $A$ is symmetric, notice that the last two terms are equal, i.e. $$x^TA (\mu_1-\mu_0) = (\mu_1-\mu_0)^T A x$$so we can write $$ \mu_0^T A \mu_0 - \mu_1^T A \mu_1 + 2x^TA (\mu_1-\mu_0)\tag{1} $$ An alternative term could be achieved by realizing that $$(\mu_1 - \mu_0)^T A (\mu_1 + \mu_0) = \mu_0^T A \mu_0 - \mu_0^T A \mu_0 +\mu_1^T A \mu_0 - \mu_0^T A \mu_1$$ again under the assumption that $A$ is symmetric, $\mu_0^T A \mu_1 = \mu_1^T A \mu_0$ hence $$(\mu_1 - \mu_0)^T A (\mu_1 + \mu_0) = \mu_1^T A \mu_1 - \mu_0^T A \mu_0 \tag{2}$$ The RHS in $(2)$ are the negative of the two terms appearing in $(1)$, pretty cool huh ? $$ -(\mu_1 - \mu_0)^T A (\mu_1 + \mu_0) + 2x^TA (\mu_1-\mu_0) $$ Again use symmetry, you can write $$ -(\mu_1 + \mu_0)^T A (\mu_1 - \mu_0) + 2x^TA (\mu_1-\mu_0) =(2x - \mu_0 - \mu_1)^T A (\mu_1-\mu_0)$$

BONUS

You say you are also interested in $$(x - \mu_0)^T A^{-1} (x - \mu_0) - (x - \mu_1)^T A^{-1} (x - \mu_1)$$Using similar steps and under the same symmetric assumption, you'll get $$(2x - \mu_0 - \mu_1)^T A^{-1} (\mu_1-\mu_0)$$