For a group $G$ with generating set $S$, we define an element of $G$ (so a vertex of the graph) to be a "dead end" if multiplying by any generator does not increase the distance from the element $1$ in the Cayley Graph $(G,S)$.
An example would be $\mathbb{Z} \times \mathbb{Z}/(2), +$ with generating set $(1,0), (1,1), (-1,0), (-1,1) (0,1)$, in which $(0,1)$ is a dead end since multiplying by any generator gives another generator (so the distance remains $ \leq 1$). However, this graph is obviously $2$-ended so it is not an answer to my question.
To elaborate on Derek's answer, one easily checks that a finitely generated group $G$ admits a locally finite Cayley graph with a dead end in the $1$-sphere if and only if it is not torsion-free.
Indeed, suppose that $G$ is not torsion-free, so it has a nontrivial finite subgroup $W$. Let $S$ be a finite generating subset (symmetric with 1), and set $T=WSW$. Then $T$ is a finite generating subset, symmetric with $1$, and $WTW=T$. So every element of $W\smallsetminus\{1\}$ is a dead end for the Cayley graph with respect to $T$.
Conversely, let $S$ be a generating subset and $s$ a dead end on the 1-sphere. We can suppose that $1\in S$ and that $S$ is symmetric (otherwise change $S$ into $S\cup S^{-1}\cup\{1\}$, which doesn't affect the Cayley graph). Since $s$ is a dead end, we have $sS\subset S$, and hence $sS=S$ by cardinality. Now let $W$ be the set of $g\in G$ such that $gS=S$. This is a subgroup of $G$, contained in $S$ and hence a finite subgroup, and it's not trivial since it contains $s$.
Edit (for the question added as a comment) In $\mathbf{Z}$ with generating subset $\{2,3\}$, $1$ is a dead end in the 2-sphere (and this is a minimal generating subset).
Similarly in $\mathbf{Z}^2$ with generating subset $\{(2,0),(3,0),(-1,1),(0,1),(1,1)\}$, the element $(1,0)$ is a dead end in the 2-sphere. This is not minimal, probably this can be arranged.
2nd Edit: here's a minimal generating subset of $\mathbf{Z}$ and an element of the 4-sphere that is a dead end. Namely $S=\{6,10,15\}$ and the element is $7$.
$7$ belongs to the $4$-ball because $7=6+6+10-15$.
The $6$ neighbors of $7$ also belong to the $4$-ball namely
That 7 is not in the 3-ball is checked by observing that the 2-sphere consists of $\{4,5,9,12,16,21,25,30\}$ and their negatives, and that none of the above 6 neighbors of 7 can be found there.