I am familiar with the uniqueness theorem that claims that given conditions on the boundary of a closed surface, if there exists a function $\varphi$ such that $\nabla^2\varphi = 0$ which also satisfy the boundary condition, it is unique. In many cases in electrostatic, infinity is used as a boundary for a closed surface.
How is this mathematically correct? Doesn't that say that every space is bounded?
Quick answers: yes and no. It is perfectly legitimate to look for solutions of the PDE $\Delta u = 0$ on the whole space such that they have a prescribed behavior at infinity. For instance, many problems require $\lim_{|x| \to +\infty} u(x)=0$.
Clearly infinity is not the boundary of $\mathbb{R}^n$, but mathematicians like to use this term in a generalized sense.