Notation - "' sign" in summation

Question

I am studying logistic regression - and i saw the following symbol: c' (in sum symbol of the denominator). What does this mean? I thought it might be all other elements of the vector except the one used in the nominator - but help much appreciated.

Now we condiser multinomial logistic regression, sometimes called a maximum entropy classifier. This is a model of the form

$$p(y=c|x, W) = \frac{\exp \left(w_c^T x \right)}{\sum_{c'=1}^C\exp \left(w_{c'}^Tx \right)}$$
[1]: https://i.stack.imgur.com/YSAGF.png

Answer 1

It's just a variable you sum on. You can replace it with any other variable that does not already appear. In general, you read the expression

$$\sum_{x=a}^b f(x)$$

as "The sum of $f(x)$ as $x$ goes from the value $a$ to the value $b$", i.e.

$$\sum_{x=a}^b f(x) = f(a) + f(a+1) + \cdots + f(b-1) + f(b).$$

In your case, the expression is

$$\sum_{c'=1}^C \exp(w_{c'}^Tx)$$

which is equal to

$$\exp(w_1^Tx) + \exp(w_2^Tx) + \cdots + \exp(w_C^Tx)$$

---

Some more explanation on the summation notation:

Notice how in the actual "meaning" of the sum, on the right hand side of the equation, the variable $x$ does not even appear. This is because $x$ is simply the variable that "runs" over all possible values between $a$ and $b$, and this is also the reason why it can be replaced by any other variable that is not yet taken. For example,

$$\sum_{i=1}^n i$$ is the same sum as

$$\sum_{j=1}^n j$$

because both sums denote the sum of the first $n$ integers.

WARNING:

I did say that you can replace the variable with any other variable that is not yet taken. This rule is crucial, and you can get into trouble if you do not follow it. For example, the sum of the $k$-th powers of the first $n$ integers is written as $$\sum_{i=1}^n i^k.$$ For example, taking $k=2,n=3$, this sum is equal to $1^2 + 2^2+3^2=14$. We can replace $i$ with something else in the expression, but not with $k$ or $n$!!!.

• Replacing $i$ with $n$ results in the expression $$\sum_{n=1}^n n^k$$ which is nonsensical: it means that $n$ runs from $1$ to itself, which makes no sense.
• Replacing $i$ with $k$ results in the expression $$\sum_{k=1}^n k^k$$ which is not the same expression as before, as now, for $n=3$, the expression is equal to $1^1 + 2^2 + 3^3 = 32$!

Answer 2

The prime is used to distinguish the summation index $c'$ in the denominator from the $c$ in the numerator, to make the expression more readable as the equivalent $$ p(y = c \mid x,W) = \frac{\exp(w_c^T x)}{\sum_{c=1}^C \exp(w_c^T x)} $$ In both cases you have to sum over all values from $1$ to $C$ in the denominator, to make that more explicit you can of course also choose another summation index, like $$ p(y = c \mid x,W) = \frac{\exp(w_c^T x)}{\sum_{d=1}^C \exp(w_d^T x)} $$