First of all, I know that there is an extremely similar question from yesterday that has been closed due to Mathematics Stack Exchange guidelines, so I can't comment and find what is incorrect in my way.
The question : find the coefficient of $(x^3y^4z)$ in $ (x+y+z)^5 (1+x+y+z)^{5}$.
We know that: $(x+y+z)^5$ = $\sum^{}_{n_1 + n_2 + n_3 = 5}$ ${5\choose n_1, n_2, n_3}$ $x^{n_1} y^{n_2}z^{n_3}$
And :
$(1+x+y+z)^5$ = $\sum^{}_{n_a + n_b + n_c +n_d = 5}$ ${5\choose n_a, n_b, n_c, n_d}$ ${x^{n_a} y^{n_b}z^{n_c}1^{n_d}}$
$ (x+y+z)^5 (1+x+y+z)^{5}$ = $\sum^{}_{n_1 + n_2 + n_3 = 5}$ ${5\choose n_1, n_2, n_3}$ $x^{n_1} y^{n_2}z^{n_3}$ $\sum^{}_{n_a + n_b + n_c +n_d = 5}$ ${5\choose n_a, n_b, n_c, n_d}$ ${x^{n_a} y^{n_b}z^{n_c}1^{n_d}}$
Which equals to:
$\sum^{}_{n_a + n_b + n_c +n_d = 5}$ $\sum^{}_{n_1 + n_2 + n_3 = 5}$ ${5\choose n_1, n_2, n_3}$ ${5\choose n_a, n_b, n_c, n_d}$ $x^{n_1} y^{n_2}z^{n_3}$ ${x^{n_a} y^{n_b}z^{n_c}1^{n_d}}$
Hence, to get the coefficient : $ n_1 =2, n_2 = 2, n_3 = 1, n_a = 1, n_b = 2, n_c = 0, n_d = 2$
So, the coefficient is ${5\choose 2, 2, 3}$ $ {5\choose 1, 2, 0, 2}$ $ = 900$ It seems incorrect because it is not in a formal way, can someone know why?
Since you want a term of total degree $8$, you might start by noting that the terms of total degree $8$ in $(x+y+z)^5 (1+x+y+z)^5 = \sum_{n=0}^5 {5 \choose n} (x+y+z)^{5+n}$ come from $ {5 \choose 3} (x+y+z)^8$.
Next, the $x^3$ term in ${5 \choose 3} (x+y+z)^8$ is ${5 \choose 3}{8 \choose 3} x^3 (y+z)^5$.
Finally, the term in $y^4 z^1$ is ${5 \choose 3}{8 \choose 3}{5 \choose 4} x^3 y^4 z^1$. So your coefficient is $$ {5 \choose 3}{8 \choose 3}{5 \choose 4} = 2800$$