Let $H \leq G$. Then,
the number of left cosets of $H$ in $G$ is the index $[G:H]$ of $H$ in $G$.
Now, I read that $[G:H]$ can either be finite or infinite. My question: if I know that $[G:H]$ is finite, would it be valid to conclude that $G$ must be finite as well? If not, would it be conclude that $H$ must be finite?
If $[G:H]$ is finite, then either $G$ is finite and $H$ is finite; or else both $G$ and $H$ are infinite. For an example of the latter, take $G$ to be the additive group of integers, $\mathbb{Z}$, and take $H$ to be the even integers $2\mathbb{Z}$. The index is $2$. (And more generally, the index of $n\mathbb{Z}$ in $\mathbb{Z}$ is $n$).
It is also possible for both $G$ and $H$ to be infinite, and for $[G:H]$ to be infinite. For example, take the additive group of polynomials with real coefficients, $G=\mathbb{R}[x]$, and take $H$ to be the subgroup of constant polynomials. Both are infinite, and $[G:H]$ is also infinite.
If $G$ is infinite and $H$ is finite, then necessarily there are infinitely many cosets of $H$ in $G$, so $[G:H]$ is infinite.
If $G$ is finite, then there are only finitely many cosets of any subgroup, so $[G:H]$ will be finite.
I will note that one can define the index to be a cardinal, so that you don’t have to just say “infinite” in the general case.