Let $x_1,x_2,\dots,x_p$ be distinct points in $\mathbb{R}^m$ (added later) in general position, where $p \geq 2$. For any $\lambda > 0$ let $K(\lambda)$ denote the $p \times p$ maxtrix whose $i,j$th entry is $\exp(-\lambda \|x_i - x_j\|^2)$ and let $\Delta(\lambda) = \det K(\lambda).$ Let $\Delta^{(k)}(\lambda)$ denote the kth derivative of $\Delta(.)$ at $\lambda.$
[This question has been changed after user @dap's comment].
- Can anything be said about the smallest positive integer $t$ such that $\dfrac{\Delta(\lambda)}{\lambda^{t}}$ tends to a non-zero limit as $\lambda \to 0^+$ in general?
It is easy to show $t \geq p-1$. We can also see that $t=p-1$ in one case. Let $M$ be the $p \times p$ matrix whose $i,j$th term is $-\|x_i - x_j\|^2$, then $t = p-1$ if the sum of all cofactors of $M$ different from $0$. It can be shown $t \geq p$ (see @Dap's comment below) when $p-1 < m+2$.
My work:
I will start with some explicit expressions for $k=2$ and $k=3$.
When $k=2$ we have $\Delta(\lambda) = \det\begin{pmatrix} 1 & \exp -\lambda \|x_1 - x_2\|^2 \\ \exp-\lambda\|x_1 - x_2\|^2 & 1\end{pmatrix} = 1 -\exp-2\lambda\|x_1 - x_2\|^2$ and we can easily see that $\dfrac{\Delta(\lambda)}{\lambda} \to 2\|x_1 - x_2\|^2 \neq 0$ as $\lambda \to 0$.
$\renewcommand{\kent}[2]{\exp-\lambda \| x_{#1} - x_{#2}\|^2}$ When $k=3$ we have $$ \Delta(\lambda) = \det \begin{pmatrix} 1 & \kent{1}{2} & \kent{1}{3} \\ \kent{1}{2} & 1 & \kent{2}{3}\\ \kent{1}{3} & \kent{2}{3} & 1 \end{pmatrix}. $$ Let $a = \|x_1 - x_2\|^2, b = \|x_2 - x_3\|^2,$ and $c=\|x_1-x_3\|^2$ .
Then $\Delta(\lambda) = 1 - \exp(-2\lambda a) - \exp(-2\lambda b) - \exp(-2\lambda c) + 2\exp -\lambda(a+b+c) = \lambda^2( 2ab + 2bc + 2ca - a^2 - b^2 - c^2) + \mathcal{O}(\lambda^3).$
So we have $\dfrac{\Delta(\lambda)}{\lambda^2}\to 2ab + 2bc + 2ca - a^2 -b^2 -c^2$ as $\lambda \to 0$ and we need to show $2ab + 2bc + 2ca - a^2 -b^2 -c^2 > 0$ where $\sqrt{a},\sqrt{b},\sqrt{c}$ are sides of a triangle.
In general $\Delta(\lambda)$ can be expanded in a power series around $0$ and so $$ \Delta(\lambda) = \Delta(0)+\Delta^{(1)}(0)\lambda+\dfrac{\Delta^{(2)}(0)}{2!}\lambda^2+\dots+\dfrac{\Delta^{(k)}(0)}{k!}\lambda^k+\dots $$ We will show below that $\Delta(0)=\Delta^{(1)}(0) = \Delta^{(2)}(0) = \dots=\Delta^{(p-2)}(0)=0.$
We have $\Delta(0)=0$ as $K(0)$ is a matrix of all ones.
More generally, if for non-negative integers $r_1,\dots,r_p$ we define $D^{(r_1,r_2,\dots,r_p)}(\lambda)$ as the determinant of the matrix whose $j$th column consists of the $j$th column of $K(\lambda)$ differentiated $r_j$ times for $1 \leq j \leq p$ (where the $0$th derivative of a function is the function itself), then we have: $$ \Delta^{(k)}(\lambda) = \sum_{\substack{r_1,\dots,r_p \geq 0\\ \sum_{j=1}^p r_j =k}} \binom{k}{r_1,r_2,\dots r_p} D^{(r_1,r_2,\dots,r_p)}(\lambda). $$ [Note: formula corrected later]
Note that $D^{(r_1,r_2,\dots,r_p)}(0)$ is $0$ if any two of the $r_j$'s are $0$. This is because all entries of $K(0)$ are 1 and $D^{(r_1,r_2,\dots,r_p)}(0)$ will become the determinant of a matrix with two columns identically equal to $1$. So if $1 \leq k \leq p-2$ we must have $\Delta^{(k)}(0)=0.$ This also implies $$ \Delta^{(p-1)}(0)/(p-1)! = D^{(0,1,1,\dots,1)}(0) + D^{(1,0,1,1,\dots,1)}(0) + \dots + D^{(0,0,\dots,0,1)}{(0)}. $$
Let $M$ be the the $p \times p$ matrix with $i,j$th term $ -\|x_i - x_j\|^2 .$ Then $ D^{(0,1,1,\dots,1)}(0)$ is the determinant of a matrix whose first columns consists of all $1$s and whose remaining columns coincide with that of $M$ and so $ D^{(0,1,1,\dots,1)}(0) $ is the sum of cofactors corresponding to the first column. Similar statements hold for other terms in the above sum and we get $\Delta^{(p-1)}(0)$ is the sum of all cofactors of $M$.