Is there any concentration inequality for the covaraince of two scalar random variables? For example, how can I found a tight upper bound for the following probability?
$$\Pr\left( {\left| {{\mathop{\rm cov}} (x,y) - \overline {{\mathop{\rm cov}} (x,y)} } \right| \le \varepsilon } \right)$$
where $\overline {{\mathop{\rm cov}} (x,y)} = E\left( {x - E(x)} \right)E\left( {y - E(y)} \right)$ is the actual covariance and ${\mathop{\rm cov}} (x,y) = \frac{1}{N}\sum\limits_{n = 1}^N {\left( {{x_i} - \hat x} \right)\left( {{y_i} - \hat y} \right)} $ is the sample covariance.
Check the paper: https://arxiv.org/abs/0811.3628, especially Lemma 1 and Lemma 2. The authors show concentration of the empirical covariance matrix under different tail conditions of the random variables.