I know that for a CW-complex $X$ with skeletons $X^n, n \geq 0$, one can identify the cellular chain group $C_n^{\text{cell}}(X)$ with the singular homology group $H_n(X^n, X^{n - 1}).$
What I don't understand at the moment, is why the boundary operator $\partial_n: C_n^{\text{cell}}(X) \rightarrow C_{n - 1}^{\text{cell}}(X)$ is equal to the composition $$H_n(X^n, X^{n - 1}) \overset{\delta}{\longrightarrow} H_n(X^{n - 1}) \overset{i_*}{\longrightarrow} H_{n - 1}(X^{n - 1}, X^{n - 2}),$$ where $\delta$ is the connecting homomorphism and $i_*$ is the induced mapping of the inclusion $i: (X^{n - 1}, \varnothing) \hookrightarrow (X^{n - 1}, X^{n - 2}).$
To be honest, I find the definition of $\partial_n$ hard to work with, so I have trouble seeing why this holds. I looked if Hatcher's Algebraic Topology had a section about it, but I did not find anything (or I simply missed it). I would really appreciate some help!
Hatcher has a whole chapter
where he defines the cellular chain complex (but does not explicitly give it a name). See p. 139. If we denote this chain complex by $C_*^{CW}$, then $C_n^{CW}(X) = H_n(X^n,X^{n-1})$ with boundaries $\partial_n$ as in your question.
I agree that this definition of $\partial_n$ is hard to work with. But let us be honest: The boundaries in the singular chain complex are defined in a very transparent way, but in general they are of no practical use for computing homology groups.
Hatcher identifies the groups $C_n^{CW}(X)$ with more "accessible" groups:
Moroever, on p. 140 he establishes the Cellular Boundary Formula whi ch allows to compute the $\partial_n$ more explicitly. However, in many cases this will not be easy because one has to determine the degree of maps $S^{n-1} \to S^{n-1}$.