Inspired by the post.
According to this paper, there are $k$-connected $k$-regular non-Hamiltonian graphs for $k=4$ and $k \ge 8$ but the other cases are not shown there.
Now I need to construct a 2-connected 5-regular non-Hamiltonian graph.
Let $P$ denote the Petersen graph, and let $P^{\prime}$ denote the graph obtained from $P$ by replacing one vertex $v$ of $P$ by the complete graph $K_3$ and making each vertex of the $K_3$ adjacent to a distinct neighbor of $v$. Hilbig proved the following theorem:
Theorem 1. (See Hilbig [1].) If $G$ is a 2-connected $k$-regular graph on at most $3 k+3$ vertices and $G \notin\left\{P, P^{\prime}\right\}$, then $G$ is Hamiltonian.
Therefore, the graph I desire should have at least 20 vertices. I searched in house of graphs, but I couldn't find a graph that meets the criteria.
Edits: I have constructed one (with 32 vertices). Perhaps it would be more meaningful to ask for a minimum graph (i.e, with minimum number of vertices) that satisfies the above properties.
The graph above is not a Hamiltonian graph, as can be inferred from the proposition below. (Note that if we remove the two vertices on the left and right sides of the graph, we will see that the remaining subgraph has five connected components.)
Proposition 1. If $G$ is a graph containing a set $S \subset V(G)$ such that $G-S$ has more than $|S|$ components, then $G$ is not Hamiltonian.
Proof. Any Hamiltonian cycle must use a distinct vertex of $S$ each time it leaves $S$, so if $G$ is Hamiltonian, $G-S$ has at most $|S|$ components.
[1] F. Hilbig, Kantenstrukturen in nichthamiltonschen Graphen, Ph.D. thesis, Technische Universität Berlin, 1986.

In the $32$-vertex example, deleting the leftmost and rightmost vertices leaves $5$ components, but it would be enough if we were left with $3$ components. We can do that instead, getting a $20$-vertex example.
Let $x$ and $y$ be the two vertices on the left and right; $G-x-y$ will have three components with vertices $a_1, \dots, a_6$, $b_1, \dots, b_6$, and $c_1, \dots, c_6$ respectively.
To define these components, begin by putting down a complete graph on each of them. Then, replace: