Let $X,Y$ be topological spaces. A function $\phi:X\to Y$ is continuous iff for any open subset $A\subseteq Y,$ the preimage $\phi^{-1}(A)$ is open in $X.$ We could similarly define a relation $\rho\subseteq X\times Y$ to be continuous iff $\rho^{-1}(A)$ is open in $X$ for any open subset $A\subseteq Y$. It can be easily seen that the composition of such relations again satisfies the condition and that the identity relation does. I had my first lecture in category theory this week so I know this gives us a category with topological spaces as objects and relations as morphisms. This is a good thing, I imagine.
For example, let's take $\rho=\{(x,y)\in \mathbb R\times \mathbb R\,|\,x^2+y^2=1\}.$ Clearly, for any open interval $(a,b)\subseteq \mathbb R,$ we have $\rho^{-1}((a,b))$ open in $\mathbb R$, and since open intervals form a basis in $\mathbb R$, this is true for any open subset $A\subseteq \mathbb R.$ So a relation that "looks continuous" is continuous in the sense defined in the previous paragraph.
However, this is hardly evidence for the notion being useful. Therefore, I would like to ask whether it makes any sense to consider such relations and whether they have been considered. I have googled "continuous binary relations" but the hits seem to be irrelevant (but perhaps the terminology has been too cryptic for me to understand they're not).
EDIT I have asked an analogous question about homomorphisms here.
There are two notions of preimage for general relations used in topology. I use the notation $y\in\phi(x)$ for $(x,y)\in\phi$. Let $\phi$ be a relation between $X$ and $Y$, aka a subset of $X\times Y$. Let $B\subseteq Y$. The upper inverse $\phi^+(B)$ of $B$ is $$\phi^+(B)=\{x\in X:\phi(x)\subseteq B\}.$$ The lower inverse $\phi^-(B)$ of $B$ is $$\phi^-(B)=\{x\in X:\phi(x)\cap B\neq\emptyset\}.$$ If $X$ and $Y$ come endowed with a topology, then we say that $\phi$ is upper hemicontinuous if the upper inverse of every open set is open, lower hemicontinuous if the lower inverse of every open set is open, and continuous if it is both upper and lower hemicontinuous.
These notions are very, very useful. Examples where continuity of relations (usually known as correspondences in this context) matters are the Maximum theorem of Berge and the Kakutani fixed point theorem. Both are fundamental tools in mathematical economics, where these notions play a big role. A great reference for these (and many other) concepts is Infinite Dimensional Analysis by Aliprantis and Border.