This question is a bit vague, but I hope its still good enough for this site.
When resolving a singularity of, say, a curve by blowing up a point, we get new variety that besides the strict transform of our curve also contains an exceptional divisor isomorphic to $\mathbb{C}\mathbb{P}^1$. Now from the explicit equations defining the blowup it is clear that we do indeed have this copy of $\mathbb{C}\mathbb{P}^1$, but I don't understand why it is there on a "deeper" level? Why is it not possible to just find a curve that is smooth and birational with our original curve except over the singular point? Or if this is somehow trivial for curves, then I ask the same question in a more general setting.
There are two different scenarios you need to be aware of here.
1) Resolution of singularities for a scheme: If you have a scheme $X$ which is singular along some locus, you can ask for a resolution of singularities $\widetilde{X}\stackrel{\pi}{\to} X$ where $\widetilde{X}$ is nonsingular and $\pi$ is proper birational. There should be no exceptional divisor here.
2) Resolution of singularities for a pair: If you have a subscheme $D\subset X$, you can ask for a resolution of the pair $(X,D)$ which means a scheme $\widetilde{X}$ and a map $\widetilde{X}\stackrel{\pi}{\to} X$ such that $\pi$ is proper birational and $\pi^{-1}(D)$ is a normal crossings divisor, ie etale-locally looks like a union of coordinate hyperplanes.
Applying 2) to an embedding of the scheme you're interested in into some variety you already know fairly well (say $\mathbb{A}^n$ or $\mathbb{P}^n$) and then taking the strict transform is usually how you compute what $\widetilde{X}$ and $\pi$ are for scenario 1). The reason you usually compute via blowups here is that we know via Hironaka that blowing up with some specified algorithm will work. (This knowledge is fairly hard-fought: resolution of singularities is difficult to prove and not so easy to understand.)
The reason why you should care about the exceptional divisor of the resolution of a pair $(X,D)$ is that the exceptional divisor tells you good information about how bad the singularities of $D$ are. There are many things you may wish to do with your singular variety, and some of them only work when certain numerical invariants (called the discrepancies) of the resolution of the pair fall in to certain ranges. Or you might be interested in telling two singular varieties apart- if you can show that these numerical invariants are different enough (and combine that knowledge with theorems about the invariants being invariants of the base variety, not just the specific resolution) then you will know that two varieties can or can't be isomorphic/birational/etc.