Suppose we have a SISO plant with the transfer function $G(s)$, and $Y(s) = G(s)U(s)$, where $Y(s)$ and $U(s)$ are the images of the output $y(t)$ and input $u(t)$, respectively. We design a SISO controller $C(s)$: $U(s) = C(s)(R(s)-Y(s))$, where $R(s)$ is the image of the reference signal $r(t)$. The standard linear system SISO design, nothing special.
Now assume that we have two possible controllers, $C_s(s)$ and $C_u(s)$, where for $C_s(s)$ all poles are either zeros or have negative real parts, and I call it an internally stable controller. And $C_u(s)$ has at least one pole with a positive real part; I call it an internally unstable controller. Suppose now that the open-loop transfer functions $C_s(s)G(s)$ and $C_u(s)G(s)$ have approximately equal phase and gain margins and the cross-over frequency, i.e., both controllers look equivalent from this point of view.
My (brief) industrial experience says that $C_u(s)$ is highly undesirable as, after fixed-point implementation, it most probably will not work in practice given all real-world nonlinearities. And $C_s(s)$ is always preferable. However, could you give me any theoretical basis for such a claim, e.g., a textbook reference? Or this claim is generally wrong?
Your experience gives the right answer. The reason to prefer stable controllers is practical, not theoretical - if something gets disconnected during operation, the separated block is stable. Other non-ideal behaviors also make the use of unstable controller non-robust, or outright dicey.
For many types of plants, no stable stabilizing controller exists. Or perhaps none gives the required performance. In such cases one has to deal with the practical difficulties created by the additional instabilities - usually by means of additional safety logic and procedures that switch components off in case of disconnection.
This is why it is desirable to design open-loop stable processes in 1st place - they can always be controlled with stable controllers. In most cases this is feasible in industry, but sometimes open-loop unstable plants have better performance - high-end combat jet planes come to mind, though I'm not an expert.