I'm trying to understand whether the following inequality is correct. Let $Y,X$ be random variables and $f(X)$, $n\times 1$-dimensional function of $X$. It is claimed that $$\begin{aligned} & a'(A'B^{-1}A)^{-1}A'B^{-1}\mathbb{E}[Yf(X)f(X)']B^{-1}A(A'B^{-1}A)^{-1}a \\ & \geq \inf_x\mathbb{E}[Y|X=x]\|a'(A'B^{-1}A)^{-1}A'B^{-1/2}\|, \end{aligned} $$ where $A,B$ are matrices and $a$ is a vector of compatible dimensions (deterministic), $\mathbb{E}[f(X)f(X)']=B$ is positive definite and $A$ is full-column rank.
How to see that this inequality is correct?
Let $$g(x)=\mathbb{E}(Y|X=x)\\ c=\inf_x g(x)\\ h^2(x)=g(x)-c$$ Then, from the right-hand side, $$||a'(A'B^{-1}A)^{-1}A'B^{-1/2}||= a'(A'B^{-1}A)^{-1}A'B^{-1}A'(A'B^{-1}A)^{-1}a\\ =a'(A'B^{-1}A)^{-1}A'B^{-1}BB^{-1}A'(A'B^{-1}A)^{-1}a\\ =a'(A'B^{-1}A)^{-1}A'B^{-1}\mathbb{E}(f(X)f'(X))B^{-1}A'(A'B^{-1}A)^{-1}a$$ So the difference between the two sides is $$a'(A'B^{-1}A)^{-1}A'B^{-1}\mathbb{E}(f(X)h(X)h(X)f'(X))B^{-1}A'(A'B^{-1}A)^{-1}a\\ =\mathbb{E}(||a'(A'B^{-1}A^{-1}A'B^{-1}f(X)h(X)||)\geq0$$