Trace of a Matrix Product.

Question

Let $A \in \mathbb{R}^{n\times m}$, and $B\in \mathbb{R}^{m\times m}$. Let $A'$ denote the transpose of $A$.

From this we know that : $A'\in \mathbb{R}^{m\times n}$ and $AB\in \mathbb{R}^{n\times m}$ and hence $ABA' \in \mathbb{R}^{n\times n}$.

The Trace of a matrix is defined as the sum of its diagonal entries. $\textbf{My Question :}$

Does anyone know bounds (using any usual matrix norm) for

• $\text{Trace}(ABA')$

• Or more generally $\text{Trace}(AB)$

Answer 1

Whenever you see a matrix trace, you should think inner product, because

$$ \operatorname{Tr}(A^T B) = \langle A, B\rangle_F = \langle A,B\rangle_{\mathbb R^m \otimes \mathbb R^n}$$

that is, the trace of the product of two matrices is equal to their frobenius inner product, which in turn is the induced inner product on the tensor product of Hilbert spaces.

Since it is an inner product, the Cauchy-Schwartz inequality applies:

$$ |\langle A, B \rangle_F |^2 \le \|A\|_F^2\|B\|_F^2$$

with equality if and only if $A$ and $B$ are linearly dependent matrices, i.e. scaler multiples of each other. In your case, we have

$$ |\operatorname{Tr}(ABA^T)| = |\operatorname{Tr}(A^TA B)| = |\langle A^TA , B\rangle_F| \le \|A^TA\|_F\|B\|_F$$

The last term can be further bounded by

$$\begin{aligned} \|A^TA\|_F\|B\|_F &\le \|A\|_F^2\|B\|_F = \Big(\sum\nolimits_i \sigma_i^2(A)\Big)\cdot\sqrt{\sum\nolimits_j \sigma_j^2(B)} \\ &\le rank(A)\cdot \sigma^2_{\max}(A)\cdot rank(B)\cdot\sigma_{\max}(B)\\ &\le m\cdot \min(m,n)\cdot \sigma^2_{\max}(A)\cdot \sigma_{\max}(B) \end{aligned}$$

Answer 2

The trace is merely the sum of the eigenvalues, so if we could find an estimate on the largest eigenvalue of $ABA'$, we could say that

$$|Tr(ABA')| \leq n\cdot|\lambda_{ABA'max}|$$

To that end we know that the eigenvalues of $ABA'$ are going to be some combination of the products of the singular values of $A$ and the eigenvalues of $B$. Take the biggest one of each to retrieve the bound:

$$|Tr(ABA')| \leq n \cdot \sigma_{Amax}^2\cdot|\lambda_{Bmax}|$$

$Tr(AB)$ is not defined because it is not a square matrix (In more abstract terms, $AB$ is not a trace-class operator).

Answer 3

$$tr(C'B)= (vec(C))'(vec(B)) \leq \bigg[(vec(C))'(vec(C)) \bigg]\bigg[(vec(B))'(vec(B)) \bigg]$$ by Cauchy Schwarz inequality. Now let $C=A'$ you get a upper bound of $tr(AB)$.

Similary for $tr(ABA')=tr(BA'A)$