Pattern Recognition and Machine Learning Exercise

Question

I am trying to solve Exercise 3.21 in Pattern Recognition and Machine Learning. The question statement can be seen below:

My attempt at the solution can be seen below:

It is obvious that I have proved the exact contradict result comparing with the problem description. Can anyone help?

Answer 1

In my opinion, you misread Exercise 3.21. It consists of two sub-problems as follows:

1. For a real, symmetric (and of course nonsingular) matrix $A$, show that $$ \frac{d}{d\alpha}\ln|A| = \operatorname{Tr}\left(A^{-1}\frac{d}{d\alpha}A\right) \tag{3.117} $$

2. Derive (3.92) by making use of (3.117).

and you seems to prove the first one.

Suppose $A\mathbf{u}=\lambda(\alpha)\mathbf{u}$ for some eigenvalue $\lambda(\alpha)$ and its corresponding eigenvector $\mathbf{u}$. Since $A$ is nonsingular, $\lambda(\alpha)\neq0$ and so $A^{-1}\mathbf{u}=\dfrac{1}{\lambda(\alpha)}\mathbf{u}$. It follows that $$ \left(A^{-1}\frac{d}{d\alpha}A\right)\mathbf{u} = A^{-1}\frac{d}{d\alpha}\bigl[\lambda(\alpha)\mathbf{u}\bigr] = A^{-1}\bigl[\lambda'(\alpha)\mathbf{u}\bigr] = \lambda'(\alpha)A^{-1}\mathbf{u}= \frac{\lambda'(\alpha)}{\lambda(\alpha)}\mathbf{u} $$ As the hint of Exercise 3.21, the determinant and the trace are expressed in terms of its eigenvalues. Thus we have $$ \begin{align*} \operatorname{Tr}\left(A^{-1}\frac{d}{d\alpha}A\right) &= \sum_i \frac{\lambda'_i(\alpha)}{\lambda_i(\alpha)} = \sum_i\frac{d}{d\alpha}\ln\lambda_i(\alpha) = \dfrac{d}{d\alpha}\sum_i\ln\lambda_i(\alpha) \\ &= \dfrac{d}{d\alpha}\ln\left(\prod_i\lambda_i(\alpha)\right) = \frac{d}{d\alpha}\ln|A| \end{align*} $$