Positive semidefiniteness of a RKHS

42 Views Asked by At

I do not get the derivation on Wikipedia which states that a RHKS is PSD. Specifically, I can see that if the kernel is PSD, then $\langle K_x, K_y \rangle$ must be PSD since $K(x, y)$ is PSD.

I do not, however, get the final step of the derivation on wikipedia:

$$ \sum_{i,j =1}^n c_i c_j K(x_i, x_j)= \sum_{i=1}^n c_i \left\langle K_{x_i} , \sum_{j=1}^n c_j K_{x_j} \right\rangle_{H} = \left\langle \sum_{i=1}^n c_i K_{x_i} , \sum_{j=1}^n c_j K_{x_j} \right\rangle_{H} = \left\|\sum_{i=1}^nc_iK_{x_i}\right\|_H^2 \ge 0 $$

How does the $j$ index completely disappear and turn into the norm only over the $i$ index? I feel like I am missing something simple here...