We have $5$ white balls, $3$ black and $2$ red in urn. We draw one after another ball with returning till we thrown red ball. Let $X$ be a number of drawn white balls. Calculate $\mathbb{E}X$.
I've started from defining $X_i$ such that $X_i=1$ when $i$-th drawn ball is white and $X_i=0$ if not. And now I have $X=\sum_{i=1}^NX_i$ when $N$ is a moment when we drawn red ball. $P(N=k)=(\frac{8}{10})^{k-1}\frac{2}{10}$ And Im not sure if I do this in right way and if yes how to end it.
On
Let $W,B,R$ denote the events that the first ball drawn is white, black, red respectively. Then:
$\mathbb{E}X=\mathbb{E}\left(X\mid W\right)P\left(W\right)+\mathbb{E}\left(X\mid B\right)P\left(B\right)+\mathbb{E}\left(X\mid R\right)P\left(R\right)\tag1$
This with
$\mathbb{E}\left(X\mid W\right)=1+\mathbb{E}X$
$\mathbb{E}\left(X\mid B\right)=\mathbb{E}X$
$\mathbb{E}\left(X\mid R\right)=0$
So (1) gives you an equation that makes it possible to find $\mathbb{E}X$:
$$\mathbb{E}X=\left(1+\mathbb{E}X\right)\cdot\frac{5}{10}+\mathbb{E}X\cdot\frac{3}{10}+0\cdot\frac{2}{10}=\frac{1}{2}+\frac{4}{5}\mathbb{E}X$$
Hence: $$\mathbb EX=\frac52$$
Your method could be salvaged, but you'd have to take into account that the $X_i$ and $N$ are dependent.
A more direct approach would be to note that the black balls are irrelevant, so this is equivalent to drawing from $5$ white balls and $2$ red balls, so you want the expected number of failures before the first success in a Bernoulli trial with success probability $\frac27$. This is $\frac72-1=\frac52$.