I was reading the page on coalgebras and it made a mention to $F$-coalgebras in the first paragraph as though $F$-coalgebras are just a specific type of coalgebras. However, I am having a difficulty understanding the explicit connection between coalgebras as in the dual notion of an algebra over a field (with a comultiplication and counit), and the $F$-coalgebras. Similarly I am not sure the explicit connection between algebras over fields and $F$-algebras.
Can anyone shed a light on the connection? Thanks!
A (co-)algebra over a field [or commutative ring] $k$ is a (co-)monoid in the monoidal category of $k$-vector spaces [$k$-modules] with the tensor product.
The crucial point in your question is finding the functor $F$ for which coalgebras are just $F$-coalgebras.
For monoids, i.e. monoids over ${\bf Set}$, we can take the functor $W:{\bf Set}\to{\bf Set},\ \ X\mapsto X^*:=\{(x_1,\dots,x_n)\mid n\in\Bbb N,x_i\in X\}$,
for which the $W$-algebras include the monoids, as each monoid $M$ naturally determines an $M^*\to M$ using the monoid operation.
Note that $W$ is just the composition of the free monoid functor ${\bf Set}\to{\bf Monoid}$ followed by the underlying set functor, and this is the general pattern.
Moreover, the definition of $F$-(co-)algebras as cited here for a functor $\mathcal C\to\mathcal C$ is just a weakening of the stronger notion of (co-)algebras for a (co-)monad $\mathcal C\to\mathcal C$.
Taking the algebras for the monad $W$ will produce exactly the monoids.
Back to your case, consider the analogoues of $W$ for vector spaces: $$F:{}_k{\bf Vect}\to {}_k{\bf Vect},\ \ \ V\mapsto\bigoplus_{n\ge 0}V^{\otimes n}$$
Then, at least we can say that coalgebras over $k$ are all $F$-coalgebras.