Let $A$ and $B$ be two matrices of order $n$ with entries in $\mathbb{R}$. $\newcommand{\lg}{\langle}$ $\newcommand{\rg}{\rangle}$
a) If $A$ and $B$ are symmetric then $$ \lg(A^{2} + B^{2})x, x \rg \geq \lg(AB+BA)x,x\rg $$ for any $x \in \mathbb{R}^{n}$ where $\lg,\rg$ means the usual inner product in $\mathbb{R}^{n}$.
hint: Consider $\lg(A-B)^{2}x,x\rg.$
b) If $A$ and $B$ are not symmetric then find a counterexample.
c) If $C$ is other matrix of order $n$ with entries in $\mathbb{R}$ and $\lg Cx,x\rg = \lg Bx,x\rg$ for all $x \in \mathbb{R}^{n}$, what can you say about $B-C$?,don't suppose $B$ is symmetric.
I tried use the hint but I don't see why help, I got $$\lg (A-B)^{2}x,x\rg = \lg (A^{2}-AB-BA+B^{2})x,x\rg = \lg (A^{2}+B^{2})x,x\rg - \lg (AB+BA)x,x\rg$$
Suppose your problem focuses on part a). The key is that since $A$ and $B$ are both symmetric, then so is $A-B$ and thus $$0\le\langle(A-B)x,(A-B)x\rangle=\langle(A-B)^2x,x\rangle, $$ and then the conclusion follows your calculation.