It is well known that Euler's lucky numbers are related to the Heegner numbers, where
\begin{align} &n^2+n+p\\ \end{align}
gives primes for $n=0,\dots,p-2$ if and only if its discriminant $1-4p$ equals minus a Heegner number. This is then true for $p=2, 3, 5, 11, 17, 41.$
There seems to be another group of numbers, namely $p=2, 3, 5, 7, 13,$ that gives primes for
\begin{align} &n^2+n-p^2\\ \end{align}
for $n=1,\dots,2p-2.$
eg, $n^2+n-13^2$ for $1\leq n\leq 24$ produces
$$-167, -163, -157, -149, -139, -127, -113, -97, -79, -59, -37, \ -13,$$$$ 13, 41, 71, 103, 137, 173, 211, 251, 293, 337, 383, 431.$$
The class numbers for the discriminants are not all the same however, and I could find no reference for this finite group, though there are many sequences that begin with these primes (Mersenne exponents, Pierpont primes, etc.).
It could, of course be coincidental, simply an example of the law of small numbers. If not though, what links these numbers if not the class numbers of the discriminants?
A Simple Answer
The Occam's razor answer would be divisors of 12, then add 1. This would lead to ${2, 3, 4, 5, 7, 13}$, where notably 4 is not included in your set.
In the case of 4 a slightly different equation is used:
$$ \frac{n^2 + n - q^2}{2} $$
e.g.: $(n^2 + n - 4^2)/2$ for $1≤n≤6$ produces
$$ {-7, -5, -2, 2, 7, 13} $$
A More Complicated Answer
I feel like an unspoken aspect of your question was the observation that $p^2 - 6$ with prime $p$ in ${2, 3, 5, 7, 13}$ results in:
$$ -2, 3, 19, 43, 163 $$
which are all, essentially, Heegner numbers (ignoring the sign).
The following is an observation that may be helpful:
If you find all solutions to $12/(x-1) = y$ where $y$ is in ${12,11,10,...,3,2,1}$ you get:
$$ {2, \frac{23}{11}, \frac{11}{5}, \frac{7}{3}, \frac{5}{2}, \frac{19}{7}, 3, \frac{17}{5}, 4, 5, 7, 13} $$
which is ${2, 3, 5, 7, 13}$ with an added even number and fractional values. When you plug these same numbers into $p^2 - 6$ you get:
$$ {-2, -\frac{197}{121}, -\frac{29}{25}, -\frac{5}{9}, \frac{1}{4}, \frac{67}{49}, 3, \frac{139}{25}, 10, 19, 43, 163} $$
Notably, the prime factorization includes every Heegner number at least once. Also, four new terms appear: ${5, 29, 139, 197}$.
This is circumstantial evidence that Heegner numbers may be related.