Hopfield Network?

Question

I have a question about the formation of weight matrix in Hopfield network: Why is not considered self-loop (i.e. $w_{ii}=0$)?

Can anyone help me in the issue? Thanks in advance!

Answer 1

The energy of a configuration $S=\{s_i : s_i= \pm1 \}$is given by

$$-E(S) = \sum_{i , j} w_{ij} s_i s_j + \sum_i \theta_i s_i $$

The cuadratic sum can be decomposed in "non-diagonal" and "diagonal" terms: $$\sum_{i , j} w_{ij} s_i s_j = \sum_{i \ne j} w_{ij} s_i s_j + \sum_{i } w_{ii} \, s_i^2 = \sum_{i \ne j} w_{ij} s_i s_j + \sum_{i } w_{ii}$$

The last equality is a consequence of $s_i^2 = 1$. Then, we see that if we modify the diagonal terms $w_{ii}$, the energy is modified by a constant amount (i.e, a value that does not depend on the configuration $s_i$). But we are not interested in the absolute value of the energy, we just want to find the configuration that attains the minimum; hence, modifying $w_{ii}$ has no relevance, hence we choose to set it to zero.