Recently, someone mentioned to me that given a function $f: X \to Y$ there are two natural functions between the powersets $P(X)$ and $P(Y)$. Namely $f: U \subset X \mapsto f(U)$ and $f^{-1}: V \subset Y \mapsto f^{-1}(V)$. Then if we consider maps between $P(P(X)), P(P(Y))$, each of the above maps induce two more, so there are four natural maps.
Thus, it seems on the face of it like there are four natural choices for morphisms of topological spaces (since a topology on $X$ is an element of $P(P(X))$). Why is it that continuous functions are the morphisms we choose and not one of the other four maps?
I understand that the theory we get from taking continuous functions as morphisms is incredibly rich and so this alone provides adequate justification. However, I am looking for a different sort of justification along the lines of "is there some property of continuous functions that immediately suggests they are the 'right' choice of morphisms between topological spaces?".
I had the same question as you when I was studying topological spaces: in particular, it annoyed me that the definition didn't look like "preserves some structure" in the sense that I'd become familiar with in abstract algebra, e.g. preserving a group operation in the case of morphisms of groups. Here were my thoughts at the time. Here are two proposals I currently have for answers to this question.
Kuratowski
There is an alternative and equivalent axiomatization of topological spaces called the Kuratowski closure axioms. Here a topology on a space $X$ is described in terms of the operation $\text{cl}$ on the power set that sends a subset of $X$ to its closure in the topology, and continuity becomes "preserves the closure operator" in the sense that $f(\text{cl}(A)) \subseteq \text{cl}(f(A))$.
Vickers
General topology is actually a kind of logic. I don't know who this insight is due to, but see Vickers' Topology via Logic for much more on this theme. In particular, the open subsets of a topological space should be thought of as axiomatizing semidecidable properties: properties that you can confirm but not necessarily disconfirm, given limited tools (e.g. finite time and precision).
For example, you can confirm whether two things are less than $5$ inches apart by measuring the distance between them to finite precision and seeing if it's less than $5$, so an open ball of radius $5$ in a metric space describes a semidecidable property, but you can't confirm whether two things are less than or equal to $5$ inches apart by measuring the distance between them to finite precision because if you get $4.99 \pm 0.2$ inches you don't know whether that's over or under $5$.
Semidecidability can be used to justify all of the topological space axioms, which is a nice exercise. For example, arbitrary unions of open sets are open because given a method of confirming whether you're in each of those open sets, you get a method of confirming whether you're in any of them by running all of the methods simultaneously and waiting for one to finish. But you only get finite intersections when you try to do the same thing for waiting for all of the methods to finish because method $n$ might take $n$ seconds finish.
Continuous functions then axiomatize "computable functions": for $f$ to be continuous means that it should be possible to compute $f(x)$ "to arbitrary precision" by computing $x$ "to arbitrary precision," where going off of the example of metric spaces "to arbitrary precision" means "to within an arbitrary open set," since it's semidecidable whether $f(x)$ is contained within an open set. In other words, to locate $f(x)$ within some open set $U$, it suffices to locate $x$ within some open set $V$. After a moment's thought you'll see that this is precisely the condition that $f^{-1}(U) = V$.
(I particularly like this justification of topological spaces and continuity because, unlike the justification coming from thinking about metric spaces, it continues to apply to spaces that aren't Hausdorff, and in fact it tells you what it means for a space to not be Hausdorff. One equivalent definition of being Hausdorff is that the diagonal $\{ (x, x) \in X \times X \}$ is closed in $X \times X$. This is equivalent to "$x \neq y$" being semidecidable, so a space fails to be Hausdorff precisely when "$x \neq y$" fails to be semidecidable.)