Prove $${Cov(X,Y)}^2 \leq Var(X)Var(Y) $$ and $${Cov(X,Y)}^2 = Var(X)Var(Y) $$ if and only if $$ Y = aX + C $$
and $$ a \neq 0 $$
I found understanding this proof helps with understanding number of concepts regarding covariance and variance, and it also involves Markov's and Boole's inequality. I wrote out a detailed proof that should be easy to follow from step to step.
Assume first that $$ Var(X) = 0 $$
By Boole's inequality we have $$ P(X \gt 0) = P( \bigcup^{n=1}_{n=\infty} (X \gt \frac {1}{n})) \geq \sum_{n=1}^\infty P(X \gt \frac {1}{n})$$
then by markov's inequality if $$ E[X] = 0 $$, we have $$ P(X \geq C) \leq \frac {E(X)}{C} = 0/C = 0 $$ for all $$ C \gt 0 $$
Hence $$ P(X \gt \frac {1}{n}) = 0 $$ For all positive n. And so $$ \sum_{n=1}^\infty P(X \gt \frac {1}{n}) = 0 = P(X \gt 0) $$ But, $$ 1 = P(X \gt 0) + P(X = 0) $$ Because $$X \geq 0 $$ And so $$ P(X = 0) = 1 $$
Since $$Var(x) = E[(X - (E[X])^2] = 0 $$ then by previous remark $$P((X - (E[X])^2=0) =1 $$ that is $$P(X=E[X]) =1 $$ and so $$ X=E[X] $$ By definition Cov(X,Y) and treating X as a constant since $$ E[X] = X $$ we have $$ Cov(X,Y) = E(XY) - E(X)E(Y) = E(X)E(Y) - E(X)E(Y) = 0 $$
For $$Var(X), Var(Y) \gt 0 $$ let $$ \rho(X,Y) = \frac {Cov(X,Y)} {\sqrt{Var(X) \cdot Var(Y)}}$$ Then if in fact $${Cov(X,Y)}^2 \leq Var(X)Var(Y) \rightarrow $$ then $$ \rho(X,Y) \geq -1 $$ and $$\rho(X,Y) \leq 1 $$ Hence $$1 \geq \rho(X,Y) \geq -1 $$
Let $$ \sigma_X, \sigma_Y $$ denote the standard deviation of X and Y. Then,
$$Var[{\frac{X}{\sigma_X}} + {\frac{Y}{\sigma_Y}}] \geq 0 $$ and treating $$ \sigma_X, \sigma_Y $$ as constants we have $$ Var[{\frac{X}{\sigma_X}} + {\frac{Y}{\sigma_Y}}]= \frac {Var(X)} {{\sigma_X}^2} + \frac {Var(Y)} {{\sigma_Y}^2} + \frac{2Cov(X,Y)}{\sigma_X \cdot \sigma_Y} $$
This is because $$ Var(A + B) = E[(A+B)^2] - E[(A+B)]^2 = E[A^2] - E[A]^2 + E[B^2] - E[B]^2 +2E[AB] -2E[A]E[B] $$
$$ = Var(A) + Var(B) + 2Cov(A,B) $$
So we have,
$$ \frac {Var(X)} {{\sigma_X}^2} + \frac {Var(Y)} {{\sigma_Y}^2} + \frac{2Cov(X,Y)}{\sigma_X \cdot \sigma_Y} = 2 \cdot (1 + \rho(X,Y)) \geq 0 \rightarrow \rho(X,Y) \geq 0 $$
Similarly, since $$ Var(A - B) = E[(A-B)^2] - E[(A-B)]^2 = E[A^2] - E[A]^2 + E[B^2] - E[B]^2 -2E[AB] +2E[A]E[B] = Var(A) + Var(B) - 2Cov(A,B) $$ Therefore,
$$ 0\leq Var[{\frac{X}{\sigma_X}} - {\frac{Y}{\sigma_Y}}] = \frac {Var(X)} {{\sigma_X}^2} + \frac {Var(Y)} {{\sigma_Y}^2} - \frac{2Cov(X,Y)}{\sigma_X \cdot \sigma_Y} = 2 \cdot (1 - \rho(X,Y) \rightarrow 0 \leq 1- \rho(X,Y) \rightarrow \rho(X,Y) \leq 1 $$
to prove linearity between X and Y if and only if $$ Cov(X, Y)^2 = Var(X) \cdot Var(Y) $$
Assume $$ Cov(X, Y)^2 = Var(X) \cdot Var(Y) $$ This implies that either $$\rho = 1$$ or $$\rho = -1 $$ Moreover, We know that $$ Var[{\frac{X}{\sigma_X}} + {\frac{Y}{\sigma_Y}}]= \frac {Var(X)} {{\sigma_X}^2} + \frac {Var(Y)} {{\sigma_Y}^2} + \frac{2Cov(X,Y)}{\sigma_X \cdot \sigma_Y} = 2 \cdot (1 + \rho(X,Y)) $$
And
$$ Var[{\frac{X}{\sigma_X}} - {\frac{Y}{\sigma_Y}}]= \frac {Var(X)} {{\sigma_X}^2} + \frac {Var(Y)} {{\sigma_Y}^2} - \frac{2Cov(X,Y)}{\sigma_X \cdot \sigma_Y} = 2 \cdot (1 - \rho(X,Y)) $$
Therefore, if $$\rho(X,Y) = -1 $$ Then,
$$ Var[{\frac{X}{\sigma_X}} + {\frac{Y}{\sigma_Y}}] = 0 $$ and so $$ {\frac{X}{\sigma_X}} + {\frac{Y}{\sigma_Y}} = C $$ where C is a constant, proving linearity between random variables X and Y.
If $$\rho =1 $$ Then $$ Var[{\frac{X}{\sigma_X}} - {\frac{Y}{\sigma_Y}}] = 0 $$ and so $$ {\frac{X}{\sigma_X}} - {\frac{Y}{\sigma_Y}} = C $$ where C is a constant, proving linearity between random variables X and Y once again.