In A Categorical Introduction To Sheaves p. 3 (2), it says that (categorically) a kernel of a homomorphism is defined by the following diagram:
I don't see how this characterizes the kernel. In particular, because (in general) for any element $a \in A$ such that $a \not\in \ker(\eta)$ the diagram does not commute. Am I having a misunderstanding about commutative diagrams?
Also, I used to have the definition of the kernel to be as it is given in Wikipedia. In particular as the limit of this commutative diagram:
This makes sense, since $0_{KY}$ maps only $K$ to $0$, as opposed to the first diagram, where all of $A$ is mapped to $0$.


The diagram is indeed not meant to be a commutative diagram.
When drawing diagrams, there are certain circumstances where one typically does not require commutativity — the most typical circumstance is a parallel pair of arrows:
$$ \bullet \rightrightarrows \bullet $$
To be a commutative diagram, the top and bottom arrows would have to be equal. This is almost never what we want, and such diagrams are very useful, so the typical convention is that commutativity is not implied by such a diagram.
However, only equation is "omitted' by convention; e.g. in a diagram
$$ \bullet \to \bullet \rightrightarrows \bullet $$
While we allow the top and bottom arrows on the right to be different, we expect the two different paths from the left to the right vertex to have same composite.
The kernel of an arrow $f : A \to B$ is the same thing as an equalizer of $f$ and the zero map $0 : A \to B$. That is what the first diagram is trying to say.
However, some people like to depict the zero arrow $0 : A \to B$ via its factorization $A \to 0 \to B$.
What has happened is that the two conventions conflict: while the diagram is intended to use the "parallel arrows are not required to be equal" convention we need to depict equalizers, rewriting the zero morphism has changed the diagram to one where this convention is not usually assumed.