Let's call this category $\mathbf{Pt}$ (if there's a standard name for it then I don't know it). I have been stuck for a few days on this problem. There is a terminal object $(\{\bullet\},\bullet)$ and products can be taken to be $(A,a)\times (B,b)=(A\times B, (a,b))$, so one might expect $\mathbf{Pt}$ to be cartesian closed. However, I strongly suspect that it's not. Intuitively, it seems to me that the functor $U:\mathbf{Pt}\to\mathbf{Set}$ sending objects $(A,a)$ to the set $A$ and sending arrows to themselves (i.e. the functor which "forgets" about the specified points) might preserve exponentials. I've solved the problem if this were true, but I don't know how to go about showing it (I don't even know if it's actually correct, honestly).
Any hints would be great.
One way to define a cartesian closed category is that it's a category with finite products where every product functor $(-) \times X$ has a right adjoint, namely the exponential functor $(-)^X$. A necessary condition for this to be the case is that $(-) \times X$ preserves colimits; in other words, that products distribute over colimits, and in particular coproducts, when they exist.
But products in pointed sets don't distribute over coproducts. For finite pointed sets $A, B, C$ the coproduct satisfies $|A \sqcup B| = |A| + |B| - 1$ and you can use this to show that $(A \sqcup B) \times C$ and $(A \times C) \sqcup (B \times C)$ have different cardinalities in general.
Alternatively, you can use the fact that pointed sets have a zero object. Here's a fun exercise: the only cartesian closed category with a zero object is the terminal category.
What is true is that pointed sets have a closed monoidal structure. The closed structure is given by defining the pointed hom-object $[A, B]$ to consist of all basepoint preserving functions $A \to B$. The basepoint in the function set is the function with a constant value equal to the basepoint of $B$. The monoidal structure is given by defining $A \otimes B$ to be the smash product, which is the quotient of $A \times B$ given by identifying all pairs $(a, b)$ such that either $a$ or $b$ is the basepoint down to a single basepoint.