A matrix group is defined to be a closed subgroup of $GL_n(\mathbb K)$ where $\mathbb K=\mathbb R,\mathbb C,\mathbb H$
In book Matrix Group: An Introduction to Lie Group Theory the author identifies $A\in GL_n(\mathbb R)$ with matrix $diag(A,1)\in GL_{n+1}(\mathbb R)$ and thus proved that $GL_n(\mathbb R)$ is a matrix group. I understand this process as $GL_{n}(\mathbb R)$ is isomorphic to a subgroup of $GL_{n+1}(\mathbb R)$ which is actually a matrix group.
And this is also applied to show $(R^n,+)\cong Trans(\mathbb R^n)$ is a matrix group.
However, the author later states in prop 1.48 that although by a homomorphism $\varphi : G \to H$ between matrix groups with $\varphi G$ closed in $H$ (which implies $\varphi G$ is a matrix group) we get an isomorphism $\bar \varphi: G/\ker\varphi\to \varphi G$, we cannot expect $G/\ker\varphi$ to be matrix group.
I don't understand why the last isomorphism cannot ensure the left side to be matrix group but the right side is. What's the difference between these isomorphism? What kind of isomorphisms would do the work? (I guess those isomorphisms that are simultaneously homoemorphisms would do, but i'm not sure)
Let's start from your definition.
$\mathbf{GL}_n(\mathbb{R})$ being closed in itself, it is a matrix group; no need to embed it somewhere. Maybe the definition you are quoting isn't the right one?
Being a closed subgroup of something isn't preserved by isomorphism: take any non-closed subgroup $H$ of a group $G$. Then $H$ is closed in itself, but not in $G$.
Therefore, if you are quoting it correctly, the book you are reading seems self-contradictory. Are you sure you didn't forget anything?
The right definition might be: a topological group is a matrix group if it is isomorphic (algebraically and topologically) to a closed subgroup of $\mathbf{GL}_n(\mathbb{K})$. If this is the case, everything seems OK, and maybe what the book is saying at the end is that if the morphism $\phi$ isn't continuous, then the quotient might not be a matrix group.