I am reading Nash's original paper "Non-cooperative games" from 1951, which could be found here:
Non-Cooperative Games, Nash (1951)
Now I have a question to criterion (2) on the second page. There it is stated
"From the linearity of $p_i(s_1, \ldots, s_n)$ in $s_i$, $$ \max_{all~r_i's} [ p_i(\mathcal{s}; r_i)] = \max_{\alpha} [ p_i(\mathcal{s}, \pi_{i\alpha})]." $$ Which essentially says, that to find an equilibrium point, you just need to consider pure strategies. Intuitively, to me it is clear that if a player $i$ doesn't play his optimal strategy (in the sense of a Nash equilibrium), then the other players could shift there probability distributions to a pure strategy which maximally exploits $i$'s non-optimal strategy. But I don't see in what sense this follows from linearity and I am unable to give a formal proof which just relies on linearity and the properties mentioned in the paper thus far?
It is not true that one has to consider only pure strategies when looking for an equilibrium point. Very simple games such as matching pennies have no equilibrium in pure strategies.
What does hold true is that the set of mixed strategy best responses of a player to some strategy profile is the convex hull of the pure strategy best responses.
All of the best responses must give rise to the same payoff and by linearity of the payoff, taking convex combinations of best responses leads to another best response. So the set of mixed strategy best responses is convex.
Now take any best response of plyaer $i$ to $\mathfrak{s}$ and two pure strategies $\pi_{i\alpha}$ and $\pi_{i\beta}$ in its support. If they give rise to different payoffs, say $p_i(\mathfrak{s},\pi_{i\alpha})>p_i(\mathfrak{s},\pi_{i\beta})$, player $i$ can increase her payoff by changing her strategy to a strategy that plays $\pi_{i\beta}$ with probability $0$ and increases the probability of $\pi_{i\alpha}$ correspondingly. This uses the linearity of the payoff function.