I'm learning about making an efficient bounding volume for a point cloud and this touches on statistics: an area of mathematics I'm very unfamiliar with.
The book talks about how to compute covariance matrices. It explains that the off-diagonal entries represent the correlation between each pair of x and y (and z if 3d) coordinates with an entry of zero indicating no correlation between the coordinates used to compute the entry:
I also found a video on YouTube about covariance which I liked and around 14:20 into the video the claim is made that the covariance will be near or equal to zero.
I decided to write some code to test this near zero covariance claim for uncorrelated variables. My code generates 5000 2D points with each $x$ and $y$ entry randomly a member of $[0,500]$. The mean point is near $(250,250)$ as expected. $\text{cov}(x,y)$ however is not typically close to zero. Examples of covariances for $x$, $y$ I'm computing are: $505.08$, $316.05$, $191.52$, $-148.70$. These would be the off-diagonal entries in the covariance matrix. How come the covariance isn't near zero?
I've triple-checked the code, but haven't completely ruled out the possibility of coding error. The C++ code can be found here.
"Near zero" is a very vague term and needs some context. Near compared to what? This is the problem with the covariance: $Cov(X,Y) = 1$ could be large or small, depending on the variance of X and Y. This is why we have correlation: it is a normalized covariance:
$Corr(X,Y)=\frac{Cov(X,Y)}{\sqrt{Var(X)Var(Y)}}$, which corrects for the overall variability of X and Y.
Since you are modelling your x,y points as uniform random variables between 0 and 500, which will each have a variance of $\sigma^2_{X\; \text{or}\;Y}=\frac{500^2}{12} \approx 20,833$ and standard deviation $\sigma_{X\; \text{or}\;Y}=\sqrt{\sigma^2_{X\; \text{or}\;Y}} = \sqrt{20,833}\approx144$, your observed covariances of 505.08, 316.05, 191.52, −148.70 need to be divided by $\sqrt{Var(X)Var(Y)} = \sqrt{20,833^2} = 20,833!$ Doing this you get the following sample correlations:
$\{0.024,0.015,0.009,-0.007\}$ -- small correlations indeed! So, I think a better thing to say is that uncorrelated variables will have correlations near 0, not their covariances.