If we have that $X_n, Y_n$ are square integrable martingales where $Y_0 \neq 0$, I'd like to show that for $n = 0,1,2,\ldots$
$$ E(X^2_{n+1}) - \frac{[E(X_{n+1}Y_{n+1})]^2}{E(Y^2_{n+1})} \geq E(X^2_n) -\frac{[E(X_{n}Y_{n})]^2}{E(Y^2_{n})} $$
A hint that was given is to consider for every $\beta$, $M^{\beta}_n = (X_n - \beta Y_n)^2$. The approach I take is to try to show that $M^{\beta}_n$ is a martingale, but I am not seeing where it could possible take us. I would greatly appreciate it if someone had an idea, thanks!
Since both $(X_n)_{n \in \mathbb{N}_0}$ and $(Y_n)_{n \in \mathbb{N}_0}$ are martingales, it follows that $(X_n-\beta Y_n)_{n \in \mathbb{N}__0}$ is a martingale for any $\beta \in \mathbb{R}$. This, in turn, implies that
$$M_n := M_n^{\beta} := (X_n-\beta Y_n)^2$$
defines a submartingale. (Note that, in general, we cannot expect that $(M_n)_{n \in \mathbb{N}_0}$ is a martingale.)
Fix $N \in \mathbb{N}$ and set
$$\beta := \beta_N := \frac{\mathbb{E}(X_{N+1} Y_{N+1})}{\mathbb{E}(Y_{N+1}^2)}.$$
(Mind that $\beta$ is well-defined since $(Y_n)_{n \in \mathbb{N}_0}$ being a submartingale and $Y_0 \neq 0$ implies $\mathbb{E}(Y_n^2)>0$ for all $n \in \mathbb{N}_0$.) Obviously,
$$\left( \frac{\beta_N}{\beta_{N-1}} -1 \right)^2 \geq 0,$$
and this is equivalent to
$$2 \frac{\beta_N}{\beta_{N-1}} - \frac{\beta_N^2}{\beta_{N-1}^2} \leq 1. \tag{1}$$
By the very choice of $\beta$ and $(M_n)_{n \in \mathbb{N}}$, we have
$$\begin{align*} \mathbb{E}(M_{N+1}) &= \mathbb{E}(X_{N+1}^2)-2 \beta_N \mathbb{E}(X_{N+1} Y_{N+1}) + \beta_N^2 \mathbb{E}(Y_{N+1}^2) \\ &= \mathbb{E}(X_{N+1}^2) - \frac{\mathbb{E}(X_{N+1} Y_{N+1})^2}{\mathbb{E}(Y_{N+1}^2)}. \tag{2} \end{align*}$$
Since $(M_n)_{n \in \mathbb{N}}$ is a submartingale, we have in particular $\mathbb{E}(M_{N+1}) \geq \mathbb{E}(M_N)$. Hence,
$$\begin{align*} \mathbb{E}(X_{N+1}^2) - \frac{\mathbb{E}(X_{N+1} Y_{N+1})^2}{\mathbb{E}(Y_{N+1}^2)} &\stackrel{(2)}{=} \mathbb{E}(M_{N+1}) \\ &\geq \mathbb{E}(M_N) \\ &\stackrel{\text{def}}{=} \mathbb{E}(X_N^2)- 2 \beta_N \mathbb{E}(X_N Y_N) + \beta_N^2 \mathbb{E}(Y_N^2). \end{align*}$$
Using again the definition of $\beta_N$, a straight-forward calculation shows
$$\begin{align*} \mathbb{E}(X_{N+1}^2) - \frac{\mathbb{E}(X_{N+1} Y_{N+1})^2}{\mathbb{E}(Y_{N+1}^2)} &\geq \mathbb{E}(X_N^2) - \frac{\mathbb{E}(X_N Y_N)^2}{\mathbb{E}(Y_N^2)} \underbrace{\left( 2 \frac{\beta_N}{\beta_{N-1}} - \frac{\beta_N^2}{\beta_{N-1}^2} \right)}_{\stackrel{(1)}{\leq 1}} \\ &\geq \mathbb{E}(X_{N}^2) - \frac{\mathbb{E}(X_{N} Y_{N})^2}{\mathbb{E}(Y_{N}^2)}. \end{align*}$$
Since $N \in \mathbb{N}$ is arbitrary, this finishes the proof.