Question: Given a one-hot random vector (a vector with one entry taking one and the rest taking zeros) $B\in\{0,1\}^L$ that follows the Categorical distribution with probability $\pi\in\mathbb{R}^L$, what is the formula for its covariance matrix $\text{Cov}[B]$?
Attempt: I started with $\mathbb{E}[B]=\pi$ and $\text{Cov}[B]=\mathbb{E}[BB^\top]-\pi\pi^\top$. Then I got stuck since I don't know how to evaluate $\mathbb{E}[BB^\top]$...
Potentially useful hint: I saw a note that presented the bounds on the multiplication of independent random one-hot vectors but gave no specific formula. I was wondering if my question (which is significantly simpler) has an explicit formula instead of bounds. Thanks.
I believe the answer should be $\text{Cov}[B]=\text{diag}(\pi)-\pi\pi^\top$. This can be gotten by evaluating $\mathbb{E}[BB^\top]$ element by element. We have $$ \mathbb{E}[BB^\top] =\mathbb{E} \begin{bmatrix} B_1B_1 & \dots & B_1B_L \\ \vdots & & \vdots \\ B_LB_1 & \dots & B_LB_L \end{bmatrix} =\text{diag}(\pi), $$ where the final equality holds true because for $l,l'\in\{1,\dots,L\}$ such that $l\neq l'$, we have $$ \mathbb{E}[B_lB_l']=0, $$ since $B_lB_l'=0$ due to the definition of the one-hot vector, and $$ \mathbb{E}[B_lB_l]=\mathbb{P}[B_lB_l=1]=\pi_l, $$ since $B_lB_l\in\{0,1\}$.