My understanding is that kernels are psd - meaning that they are symmetric and have positive eigenvalues.
But, are there examples of a kernel that is symmetric and positive in the sense that $k(x,x') \geq 0$, but is not PSD? Or must all kernels be at least conditionally PSD?
Conversely, is there a kernel that is PSD but does not satisfy $k(x,x) \geq 0$ for all $x, x' \in \mathcal{X}$?
For a kernel that is symmetric and nonnegative but not positive semidefinite, we just need a $2\times 2$ matrix $K$ with $K_{ij}\geq 0$ which is symmetric but has a negative eigenvalue. Consider $$ K = \begin{bmatrix} 1 & 2 \\ 2 & 2 \end{bmatrix} $$ which has eigenvalues $\tfrac 12 (3 \pm \sqrt{17})$.
For a kernel that is symmetric and positive semidefinite but not nonnegative, we just need a $2\times 2$ matrix $K$ with $K_{ij}\not\geq 0$ which is symmetric but has a negative eigenvalue. Consider $$ K' = \begin{bmatrix} 1 & -1 \\ -1 & 1 \end{bmatrix} $$ which has eigenvalues $2,0$.