An affine group scheme $G$ over the field $k$ is a functor $$G:Alg_k\to Grp$$ that is isomorphic (as a functor) to a functor $$h^A=hom_{alg_k}(A,-),$$ where $A$ is a hopf-algebra (which gives the group structure to $G$. My notes don't require that this is a functor from hopf k-algebras, yet they still say that by the Yoneda lemma $G\to H$ (a natural transformation of functors) corresponds to a hopf-algebra homomorphism $F[H]\to F[G]$. But this doesn't actually follow? $G\cong h^{F[G]}$ and $H\cong h^{F[H]}$ then we have by Yoneda that $$Nat(h^{F[G]},h^{F[H]})\cong h^{F[H]}(F[G])=hom_{alg_k}(F[H],F[G]).$$
I understand that the hopf-algebra structure is only needed for $A$ in $h^A$ since this gives group structure to the a priori hom-set, but then Yoneda doesn't provide a hopf-algebra homomorphism. How do I fix this?
This is just a slightly elaborated version of Roland's comment:
It's not correct that $\textit{all}$ natural transformations $h^A \to h^B$ correspond to Hopf-algebra homomorphisms.
However, natural transformations that are morphisms of $\textit{group schemes}$ do. $h^A \to h^B$ corresponds to $k$-algebra morphisms $B \to A$ by Yoneda, as you wrote down as well.
What could be meant is the following though:
A morphism $G \to H$ of $\textit{group}$ schemes is a natural transformation of functors $_k\mathsf{Alg} \to \mathsf{Grp}$.
Such a natural transformation $\eta : G \to H$ gives rise to a natural transformation $f_{\eta} : h^{F[G]} \to h^{F[H]}$, which in turn (as you explained in the question) corresponds to $k$-algebra morphisms $F[H] \to F[G]$ by Yoneda.
The question is: Which natural transformations $h^{F[G]} \to h^{F[H]}$ come from a natural transformation $G \to H$ of functors $_k\mathsf{Alg} \to \mathsf{Grp}$?
And the answer to that would be: Its those transformations that correspond to $k$-Hopf algebra homomorphisms $F[H] \to F[G]$.
It's not only by Yoneda though. It's because natural transformations $G \to H$ have to be $\textit{group homomorphisms}$ on the level of points. One can translate those properties by stating the group axioms via commutative diagrams and by "reversing the arrows" (as the Yoneda embedding we are talking about is contravariant).
An even more categorical point of view: We have the category $_k\textsf{Alg}$ of $k$-Algebras and $k$-Hopf algebras are exactly the cogroup objects of this category.
Then there is the presheaf category $[_k\textsf{Alg}^{op},\textsf{Set}]$. Yoneda gives you a fully faithful (contravariant) embedding $_k\textsf{Alg} \to [_k\textsf{Alg}^{op},\textsf{Set}], A \mapsto h^{A}$.
The full subcategory consisting of the essential image of this functor is the category $\textsf{AffSch}_{/k}$ of affine schemes over $k$ and because Yoneda is fully faithful and we took the essential image, this gives a (contravariant) category equivalence $_k\textsf{Alg} \to \textsf{AffSch}_{/k}$.
Hence cogroup objects of $_k\textsf{Alg}$ - that is: $k$-Hopf algebras - correspond to group objects of $\textsf{AffSch}_{/k}$ - affine group schemes.