If I have a non-compact set, like the one depicted in the image, would it be correct to say that the dashed line is the boundary of the set?
What would be the closure of this particular set?
I don't understand how to handle "infinity" when talking about this kind of concepts. Any suggestions would be greatly appreciated.
Example Plot:

(Ignoring that the "dashed line" is not actually a line.)
Looking at the various ways to define "boundary" (I am thinking of the third one on its wikipage here) you may note that, in the image you have provided (and assuming you are using the standard Euclidean topology on $\mathbb{R}^2$) it is true for any given point on the dashed line that every neighborhood around it contains points in the shaded set and not in the shaded set; so, yes: the dashed line is in the boundary.
No, not quite, at least the way I am looking at the attached image, which is that the left and right "sides" (located at $x=\pm 5$) are also part of the boundary. So, saying that the dashed line is the boundary of the set is not quite right; rather, it is a subset of the boundary of the set. If the image rather "continues" in both directions off to $x = \pm \infty$, then yes: the dashed line is precisely the boundary.
For the image provided, make the dashed line into a hard line and you have the closure, which is itself equal to the union of the interior of the set and the boundary of the set.