Let $F: \mathbf{A} \to \mathbf{B}$ be any functor and $B$ an object of $\mathbf{B}$. We say the pair $(A,\epsilon)$, with $A$ and object of $\mathbf{A}$ and $\epsilon: FA \to B$ is a morphism of $\mathbf{B}$ is cofree over B with respect to $F$ just in case $\epsilon:FA \to B$ has the universal property that given any morphism $f:FA^\prime \to B$ (with $A^\prime$ an object of $\mathbf{A}$) there exists unique $A$-morphism $\psi: A^\prime \to A$ such that the appropriate diagram commutes, namely $$ \epsilon \circ F\psi = f. $$
The above definition is taken word for word from Arbib. He goes on to say that if the $F$ is $\cdot \times X_0: A \mapsto A\times X_0$, then the corresponding $A$ should be $B^{X_0}$ and $\epsilon$ should be the evaluation map, namely $$ \epsilon: B^{X_0}\times X_0 \to B, \ \ \epsilon (f,x):=f(x). $$
My question is this, given the co-universal definition, how does one think in order to construct the appropriate $A$ and $\epsilon$. I am interested in the thought process that leads someone to "discovering" both the set $A$ and the map $\epsilon$.
For example if I take the functor $F$ to be from the category of vector spaces, and let it assign $F: V \to V\otimes V$, then what would the pair $(A,\epsilon)$ be in this case, that would satisfy the co-free property?
I agree with Derek Elkins' suggestion in the comments: If you want to have intuition for what cofree morphisms are, since cofree morphisms are the counits of adjunctions, one way to go is to study the constructions in the adjoint functor theorem, though I also suggest just looking at more examples of adjoints in general.
One powerful part of category theory is just how intuitive some of the definitions are, once you get used to what they mean. In particular, the notion of natural transformation describes accurately what it means to "naturally" have arrows between $F(a)\to G(a)$ for all $a$ when $F,G$ are functors. Thus one way to understand certain definitions in category theory is to take advantage of its intuitive notions, such as naturality.
For example, let's rephrase the definition of cofree that you gave. A cofree arrow $f: FA \to B$ gives a correspondence between arrows from $FA'$ to $B$ and arrows $A' \to A$ via the universal property. In other words, there is a natural bijection between the functors Hom$(-,A)$ and Hom$(F-,B)$. In a Yoneda Lemma-like manner, we can look at the arrow corresponding to the identity in Hom$(A,A)$ to get an arrow in Hom$(FA,B)$ that is our original cofree arrow. Thus we see that a cofree arrow is exactly given by a natural bijection Hom$(-,A) \cong$ Hom$(F-,B)$.
Thus finding the cofree arrow amounts to finding an $A$ such that arrows from $A'$ to $A$ "naturally correspond" to arrows from $FA'$ to $B$.
For example in the case $F$ is $- \times X$, the condition is Hom$(A',A) \cong$ Hom$(A'\times X,B)$. We would like set such that a function $f: A'\times X \to B$ "can be thought of as" a map $A' \to A$.
If $f(a,x) = b$, then we would like to repackage this function in a way to make it only a function of $A'$ while not losing or gaining information. Then it shouldn't be too surprising that the correct object to do this is $B^X$, as we can send $a$ to the function that sends $x$ to $f(a,x)$ which gives the natural correspondence we want.
Let's try this. We would like to find a vector space W such that Hom$(V',W) \cong$ Hom$(V'\otimes V',V)$. Note now that the set of homomorphisms is no longer just a set, but a vector space. If we expect a nice, truly natural isomorphism to exist, we had better expect it to be an isomorphism of vector spaces, not just of sets.
But this leads to a problem. If we let $V,V'$ be finite dimensional, then we have dim$($Hom$(V'\otimes V',V))$ = dim$(V')^2$dim$(V)$ whereas Hom$(V',W)$ has dimension dim$(V')$dim$(W)$. Now certainly we can't choose the dimension of $W$ such that $n^2$dim$(V)$ = $n$dim$(W)$ for all $n$, so we shouldn't expect $W$ to exist.
Here is an example that does work.
We can consider the tensor product functor in the category of vector spaces, $(-)\otimes V$, and ask if there is a cofree arrow to $W$. In other words we would like to find a $W'$ such that Hom$(V',W') \cong$ Hom$(V'\otimes V,W)$. However one definition of the tensor product of two vector spaces $V',V$ is a vector space such that bilinear maps $V\times V' \to W$ correspond to linear maps $V\otimes V' \to X$. However, a bilinear map is nothing more than a linear map to a vector space of linear maps, or an element of Hom$(V',$Hom$(V,W))$, so we have an isomorphism Hom$(V',$Hom$(V,W)) \cong$ Hom$(V'\otimes V,W)$, and Hom$(V,W)$ is then the $W'$ that we want.
We can now recover the cofree arrow by considering the map corresponding to the identity, which is the evaluation map Hom$(V,W)\otimes V \to W$.
The reason this works is the tensor-hom adjunction, which characterizes the tensor product, and is why tensor products are so useful and important.
I hope these examples gave at least a little insight into how to think about the existence of cofree arrows. I will just note once again that the majority of good examples will be examples of adjoints, which are good to look into to better understand this.