One matrix is diagnolized by orthonormal basis of another matrix

Question

Let $A\in \mathbb R^{n\times n}$. Suppose $B$ is symmetric and positive definite, and \begin{equation}\label{eq:sym} A^TB=BA, \end{equation} then $A$ is diagnolizable by a B-orthonormal basis. Therefore, there exist a B-orthogonal matrix $U=(u_1,\cdots,u_n)$ with \begin{align}\label{eq:orthnor} U^TBU=I_n, \end{align} and a diagonal matrix $D=\text{diag}(\lambda_1,\cdots,\lambda_n)$, such that \begin{align}\label{eq:diag} AU=UD, \end{align}

An important implication is that for every vector $x\in \mathbb R^n$, we have the expansion \begin{align} x=UU^TBx, \quad Ax=UDU^TBx. \end{align}

Questions:

1. Which techniques are related in functional analysis or linear algebraic?
2. How to find a B quickly? even it is not unique.

Answer 1

This is conceptually simple but a bit tedious. Working over $\mathbb C$, here's the proof of the claim
$A^*B=BA$ and $ U^*BU=I_n, $

First consider the special case of $A =R$ an upper triangular matrix.
Then $R^*LL^*=R^*B = BR=LL^*R $
where $B=LL^*$ by Cholesky factorization
$\implies L^{-1}R^*L= L^*R(L^*)^{-1}$
via multiplication on left by $L^{-1}$ and on the right by $(L^*)^{-1}$

since upper triangular matrices form a (semi)group we conclude that the LHS is lower triangular and the RHS is upper triangular therefore $L^*R(L^*)^{-1}=D$ a diagonal matrix.

For the general case:
$A=QRQ^*$ by Schur Triangularization.
$A^*B=BA$
$\implies R^*B'= (Q^*A^*Q) (Q^*BQ) = Q^*A^*BQ=Q^*BAQ= (Q^*BQ)(Q^*AQ)=B'R$
and the argument runs as before with $B':=LL^*$, its Cholesky decomposition. So $(L^*)^{-1}DL^*=R$ and multiplying on the left by $Q$ and right by $Q^*$ tells us $A=UDU^{-1} = Q(L^*)^{-1}DL^*Q^*$. In your notation $U^{-1}= (QL)^*=L^*Q^*$. Similarly $B=QB'Q^*=Q(LL^*)Q^*=(QL)(QL)^*$.

Which techniques are related in functional analysis or linear algebraic?

Really this comes down to manipulating congruence and triangular matrices.

How to find a B quickly? even it is not unique.

I don't know about 'quickly' but for diagonalizable $A = UDU^{-1}$, (with a slight overload of notation) run QR factorization on $U= Q'R'\implies U^{-1}=(R')^{-1}(Q')^{*}$ so $Q:=Q'$ and $L^*:=(R')^{-1}$ and of course $B=LL^*$. Note: that if $A$ has all real eigenvalues then $U, Q', R'\in \mathbb R^{n\times n}$, and we recover the original claim which was over reals.