I have read both my textbook and the wikipedia page but there is a nuance about the notation that I am lost about.
The variance is defined as:
$E((X-\mu)^2)$
Assume we are on a finite case.
Does that mean that the expansion is:
$\sum (x_i-\mu)^2p(x_i)$
Or:
$\sum (x_ip(x_i)-\mu)^2$
On
It means the first one. In general for the discrete case for a function $g$ of the random variable $X$
$$ \mathbb{E}[g(X)] = \sum_i g(x_i)p(x_i) $$
if you replace $g(X) = (X - \mu)^2$, then
$$ \mathbb{E}[(X -\mu)^2] = \sum_i (x_i -\mu)^2p(x_i) $$
On
The first one is correct.
$$E[(X-\mu)^2]= \sum (x_i-\mu)^2p(x_i)$$
If we approach expectation values as weighted sums, we point-wise multiply a weight $w(x_i)$ against the distribution $p(x_i)$ and then sum. This fits the form above.
$$E[w(X)]= \sum w(x_i)p(x_i)$$
Note that your second example results in
$$\begin{align} \sum (x_ip(x_i)-\mu)^2 &= \sum x_i^2p(x_i)^2 - 2\mu\sum x_ip(x_i) + \mu^2\sum 1 \\ &= \sum x_i^2p(x_i)^2 - 2\mu E[X] + N\mu^2 \\ &= E[X^2p(X)] - 2\mu E[X] + N\mu^2 \end{align}$$
where the first term involves the expectation value of $X^2p(X)$ and I assume the sum runs from $i=1$ to $N$.
You want the average squared deviation. First compute the average $\mu$ by the usual formula:
$$ \mu = \sum_{x \in \Omega} xp(x) $$
if $X$ is a discrete random variable, or
$$ \mu = \int_{\Omega} x \ dF(x) = \int_{\Omega} x f(x) \ dx $$
where $f$ is the density and $F$ is the CDF.
Once you have this, you'll now find the deviations $x - \mu$, square them all, and find their average. That is, you'll compute
$$ \sigma^2 = \sum_{x \in \Omega} (x - \mu)^2p(x) $$
which gives the variance. (A similar formula is available for continuous random variables.) I challenge you to use this to derive the shortcut formula $\sigma^2 = E(X^2) - \mu^2$ where $E(x^2)$ is the average of the squares!