A bag contains 52 balls: 13 red, 13 blue, 13 green and 13 yellow. I draw 5 balls without replacement. $X$ is the number of red balls I have drawn and $Y$ is the number of blue balls I have drawn. What is $Cov(X,Y)$?
I first want to compute $E(X)$ and $E(Y)$ and my idea is to express $X$ and $Y$ as sums of 5 indicators, where each indicator represents whether the $i$th ball is red/blue, that is $X = \sum_{i=1}^{5} 1$(the $i$th ball is red) and $Y = \sum_{i=1}^{5} 1$(the $i$th ball is blue). Is that a good idea? However, I don't know how to compute the probability of each indicator.
Yes, that is the way to go. Let $X_i$ be the indicator that ball $i$ is red, and $Y_i$ the indicator that it is blue. Thus:
$$X=\sum_{i=1}^5 X_i\\Y=\sum_{i=1}^5 Y_i$$
The expectation of $X_i$, which is also the marginal probability that ball $i$ is red, when each of the $52$ balls are equally probable to be that ball and $13$ among them are red, is clearly:
$$\begin{align}\mathsf E(X_i) &= \mathsf P(X_i=1) \\& = \dfrac{13}{52}&&=\dfrac 14\end{align}$$
Likewise $\mathsf E(Y_i) = 1/4$ too.
Obviously a single ball cannot be both red and blue, so $\mathsf E(X_i Y_i)=0$.
Now, all you need is $\mathsf E(X_iY_j)$ when $i\neq j$. The joint probability that ball $i$ is red and ball $j$ is blue.
Then use the formula for covariance ...