Can someone explain the definition of a subsheaf here, I am somewhat confused about the restriction map, I can't understand why $\rho_{\mathscr{K},U,V}(\mathscr{K}(U)) \subseteq \mathscr{K}(V)$ for the restriction map?
Also the last part seems a little strange, the quotient sheaf $\mathscr{F}/ \mathscr{K}$ is defined to be the sheaf associated to the presheaf $U \mapsto \mathscr{F}(U) / \mathscr{K}(U)$. What is meant here, is it a sheaf or presheaf?
If it is not true that $\rho_{\mathscr{F},U,V}(\mathscr{K}(U)) \subseteq \mathscr{K}(V)$, then you just don't have a subsheaf. In other words, a subsheaf is given by a family of subgroups $\mathscr{K}(U)\subseteq\mathscr{F}(U)$ such that $\rho_{\mathscr{F},U,V}(\mathscr{K}(U)) \subseteq \mathscr{K}(V)$ is always true, and then you make this family of subgroups into a presheaf by restricting the restriction maps from $\mathscr{F}$. (To have a subsheaf, this presheaf must then additionally actually be a sheaf.)
For the quotient sheaf, there is a presheaf $Q$ defined by $Q(U)=\mathscr{F}(U)/\mathscr{K}(U)$. This presheaf may or may not be a sheaf. The quotient sheaf $\mathscr{F}/\mathscr{K}$ is then defined as the sheafification of this presheaf $Q$.