I am interested in understanding the proof the there is no unitary finite dimensional representation of $SL(2,\mathbb R)$ from Knapp's book. But the argument is hard to follow after some time. I have come up with the following argument. I want to know if it is correct.
Take the Cartan decomposition $\mathfrak{sl}(2,\mathbb R)$ as $\mathfrak f \oplus \mathfrak p$ where $\mathfrak f$ and $\mathfrak p$ consist off skew-Hermitian and Hermitian matrices. If $\Phi:SL(2,\mathbb R)\to GL(V)$ is finte dimensional unitary representation, then the diffenential $d\Phi$ is a Lie algebra homomorphism. Furthermore as $\mathfrak{su}(2)=\mathfrak f\oplus i\mathfrak p$, $d\Phi$ induces a representation of $\mathfrak{su}(2)$ as $\phi(X\oplus iY)=d\Phi(X)\oplus id\Phi(Y).$ As $SU(2)$ is simply connected and connected we can have a representation $\Psi:SU(2)\to GL(V)$ with $d\Psi=\phi$ and $\Psi$ is unitary. Therefore, $\Psi(g)^*=\Psi(g^{-1})=\Psi(g^*)$ for all $g\in SU(2).$ Therefore, we must have $\phi(X)^*=\phi(-X)=\phi(X^*)$ for all $X\in\mathfrak{su}(2).$ Hence for $X=X^*$ we have $\phi(X)=0.$ Therefore, $d\Phi(\mathfrak p)=0.$ Since $[\mathfrak f,\mathfrak p]=\mathfrak{sl}(2,\mathbb R)$ we must have $d\Phi=0.$ Hence $\Phi$ is trivial. But I am afraid that I have not used anywhere that $\Phi$ is unitary!! Where did my argument go wrong?
The problem about your argument is being unitary for a representation just requires that some Hermitian inner product is preserved. Usually this is not a big deal, since there is only one such inner product on a finite dimensional vector space up to isomorphism. Hence, up to conjugation, you can always assume that a fixed inner product is preserved. This does not work any more if two groups are involved. You start with a unitary representation of $SL(2,\mathbb R)$ on $V$, so implicitly,you fix an inner product. Then you construct a representation of $SU(2)$ (which actually would not be necessary, you could stay at the level of Lie algebras right away). But when you say that that representation of $SU(2)$ is unitary, then this does certainly not mean that it preserves the given inner product. (This gets hidden even more, since you don't use an inner product explicitly but only the $*$ operation on matrices - which has the inner product in its definition.) Your line of argument probably can be made to work but I think this gets pretty complicated and might need classification of representations.
There is a simple argument based on conjugacy classes as follows: Suppose that $\Phi:SL(2,\mathbb R)\to U(N)$ is a (continuous) homomorphism. Then look at the matrices $A_t:=\begin{pmatrix} 1 & t \\ 0 & 1\end{pmatrix}$. For $t\neq 0$ they all have the same Jordan normal form (namely the matrix for $t=1$). On the other hand, for $t\to 0$, they tend to the identity map. The first statement means that there is $B_t\in SL(2,\mathbb R)$ such that $A_t=B_tA_1B_t^{-1}$ and hence $\Phi(A_t)=\Phi(B_t)\Phi(A_1)\Phi(B_t)^{-1}$. Now consider the conjugacy class $C:=\{U\Phi(A_1)U^{-1}:U\in U(N)\}$\subset U(N). This is a continuous image of $U(N)$ and thus compact and hence a closed subset. By continuity of $\Phi$, $\mathbb I=\Phi(A_0)$ is the limit for $t\to 0$ of $\Phi(A_t)$. We conclud that $\mathbb I\in C$, so $\mathbb I=U\Phi(A_1)U^{-1}$ for some $U\in U(N)$. This implies $\Phi(A_1)=\mathbb I$ and hence $\Phi(A_t)=\mathbb I$ for all $t$. Differentiating, you see that $\phi=\Phi'$ vanishes on $\begin{pmatrix} 0 & 1 \\ 0 & 0\end{pmatrix}$, which easily implies that $\phi=0$.