It is a well known example that the quotient of $H$ by $N$ is not a matrix Lie group (I've already seen proofs it is a Lie group, I am only interested in it not being a matrix group), where:
$$H = \left\{\left(\begin{matrix} 1 & a & b \\ 0 & 1 & c \\ 0 & 0 & 1 \end{matrix}\right) : a,b,c \in \mathbb{R} \right\}$$
$$N = \left\{\left(\begin{matrix} 1 & 0 & x \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{matrix}\right) : x \in \mathbb{Z} \right\}$$
I am trying to prove this is not a matrix Lie group by proving it has no faithful representation. Given a representation $\varphi : H/N \to GL(V)$, we need some nontrivial kernel, then we are done. My candidate kernel elements are:
$$T = \left\{\left(\begin{matrix} 1 & 0 & x \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{matrix}\right)N : x \in \mathbb{R} \right\}$$
$T$ is isomorphic to $\mathbb{R}/\mathbb{Z} = S^1$, so if we restrict $\varphi$ to $T$, we get a representation of $S^1$, which splits into irreducibles in a very well-known way. Is there something about this that means we only get the trivial representation for $T$, so it lies in the kernel of $\varphi$? This really seems like it is true and I hope it is, any help finishing this argument would be appreciated.
In such a representation of the quotient, the restriction of the representation to $S^1$ would allow us to decompose the reps into ones in which the group element with only $z$ in the corner would act as multiplication by the the scalar $e^{2\pi inz}$ (these are the reps of $S^1$).
Therefore, at the lie algebra level, the matrix with $1$ in the corner and $0$s elsewhere would act diagonally, and we may decompose $V$ into eigenspaces $V_{\lambda}$. The image of the representation in $\mathfrak{gl}(V)$ is spanned by $X,Y,Z$ satisfying $[X,Y]=Z$ and $Z$ central. Since $X,Y$ commute with $Z$, they must act block-diagonally with respect to the eigendecomposition $V=\bigoplus_{\lambda}V_{\lambda}$ (which is with respect to $Z$). Let $X_{\lambda},Y_{\lambda}$ be the restrictions of the linear operators $X,Y$ to the subspaces $V_{\lambda}$. Then the trace $\mathrm{tr}([X_{\lambda},Y_{\lambda}])$ must be $(\dim V_{\lambda})\lambda$ since it is multiplication by $\lambda$ but $\mathrm{tr}(X_{\lambda}Y_{\lambda}-Y_{\lambda}X_{\lambda})=0$ (since $\mathrm{tr}(XY)=\mathrm{try}(YX)$ is an identity). Thus $\lambda=0$.
We conclude $Z$ acts as $0$ so $T$ must act as $I$, and so all of $T$ is in the kernel.