Consider the following simultaneous move game:
All random vectors below are defined on the same probability space $(\Omega, \mathcal{F}, P)$
Players' compute expectations correctly using $P$
$n$ players
Players indexed by $i\in \{1,...,n\}$
$\text{I}_i$ is player $i$'s information set; $\text{I}_i$ is a random vector with support $\mathcal{I}_i$; $I\equiv (\text{I}_1,..., \text{I}_n)\equiv (\text{I}_i, \text{I}_{-i})$
$\mathcal{D}_i$ is player $i$'s action space
$s_i: \mathcal{I}_i \rightarrow \mathcal{D}_i$ is player $i$'s strategy (we assume that only pure strategies are possible)
$D_i=s_i(\text{I}_i)$ is player $i$'s action, with support $\mathcal{D}_i$
When playing, player $i$ cares about other pre-determined features $Z_i$; $Z_i$ is a random vector with support $\mathcal{Z}_i$
$\pi: \mathcal{D}_i \times \mathcal{D}_{-i}\times \mathcal{Z}_i \rightarrow \mathbb{R}$ is player $i$'s profit function
The equilibrium concept is pure strategy Bayesian Nash equilibrium (PSBNE) defined as follows: player $i$'s best response set given $\text{I}_i, s_{-i}$ is defined as $$ \begin{aligned} \text{BR}_i(\text{I}_i, s_{-i}) &\equiv \text{argmax}_{s_{i}} {E}_{\text{I}_{-i},Z_i}\Big( \pi(s_i(\text{I}_i), s_{-i}(\text{I}_{-i}), Z_i)\Big| \text{I}_i, s_i, s_{-i} \Big) \end{aligned} $$ (assume $\max$ exists for simplicity; ${E}$ denotes expectation and the subscript of $E$ clarifies with respect to which random variables we are taking the expectation)
Hence, $s\equiv (s_1,..., s_n)$ is a PSBNE if $$ s_i(\text{I}_i)\in \text{BR}_i(\text{I}_i, s_{-i}) \text{ }\forall i \in \mathcal{N} \text{, a.s. } \text{I} $$ or, equivalently, if $$ \begin{aligned} {E}_{\text{I}_{-i},Z_i}\Big( \pi(s_i(\text{I}_i), s_{-i}(\text{I}_{-i}), Z_i)\Big| \text{I}_i, s_i, s_{-i} \Big) \geq \max_{\tilde{s}_i} & {E}_{\text{I}_{-i},Z_i}\Big( \pi(\tilde{s}_i(\text{I}_i), s_{-i}(\text{I}_{-i}), Z_i)\Big| \text{I}_i, \tilde{s}_i, s_{-i} \Big) \\ &\forall i \in \mathcal{N} \text{, a.s. } \text{I} \end{aligned} $$ (Notice that a PSBNE may not exist or there may be more than one PSBNE)
A book I am reading claims at this point that a NECESSARY CONDITION FOR PSBNE is that
$$ \begin{aligned} (\star)\text{ }{E}_{D_i, D_{-i},Z_i}\Big( \pi(D_i, D_{-i}, Z_i)\Big| \text{I}_i \Big) \geq \max_{\tilde{d}\in \mathcal{D}_i} {E}_{D_{-i},Z_i}\Big(& \pi(\tilde{d}, D_{-i}, Z_i)\Big| \text{I}_i \Big)\\ & \forall i \in \mathcal{N} \text{, a.s. } \text{I}_i \end{aligned} $$
Question: Why is that the case? A proof may be the following
(1) If $s_i$ denotes PSBNE played by player $i$, then $$ \begin{aligned} &E_{D_{-i},Z_i}\Big( \pi(s_i(\text{I}_i), D_{-i}, Z_i)\Big| \text{I}_i, s_i \Big) \\ &\geq \max_{d\in \mathcal{D}_i} E_{D_{-i},Z_i}\Big( \pi(D_i, D_{-i}, Z_i)\Big| \text{I}_i, D_i=d\Big) \text{ } \forall s_i\text{, }\text{ a.s. } \text{I}_i \end{aligned} $$ $$ \Updownarrow $$ $$ \begin{aligned} &E_{D_{-i},Z_i}\Big( \pi(s_i(\text{I}_i), D_{-i}, Z_i)\Big| \text{I}_i, s_i) \Big) \\ &\geq E_{D_{-i},Z_i}\Big( \pi(D_i, D_{-i}, Z_i)\Big| \text{I}_i, D_i=d\Big) \text{ }\forall d \in \mathcal{D}_i\text{, }\forall s_i\text{, }\text{ a.s. } \text{I}_i \end{aligned} $$ $$ \Updownarrow $$ $$ \begin{aligned} &\sum_{d_{-i}, z_i} \Big[ \pi(s_i(\text{I}_i), d_{-i}, z_i)\times \mathbb{P}(D_{-i}=d_{-i}, Z_i=z_i| \text{I}_i, s_i) \\ &\hspace{-1.2cm}- \pi(d, d_{-i}, z_i)\times \mathbb{P}(D_{-i}=d_{-i}, Z_i=z_i| \text{I}_i, D_i=d) \Big]\geq 0 \text{ }\forall d \in \mathcal{D}_i\text{, }\forall s_i\text{, }\text{ a.s. } \text{I}_i \end{aligned} $$
(2) By simultaneity of interactions and by the fact that $Z_i$ is assigned prior to the game, $(D_{-i}, Z_i)| \text{I}_i, D_i \sim (D_{-i}, Z_i)| \text{I}_i$, so that above becomes $$ \begin{aligned} &\sum_{d_{-i}, z_i} \Big[ \pi(s_i(\text{I}_i), d_{-i}, z_i)\times \mathbb{P}(D_{-i}=d_{-i}, Z_i=z_i| \text{I}_i) \\ &- \pi(d, d_{-i}, z_i)\times \mathbb{P}(D_{-i}=d_{-i}, Z_i=z_i| \text{I}_i) \Big]\geq 0 \text{ }\forall d \in \mathcal{D}_i\text{, }\forall s_i\text{, } \text{ a.s. } \text{I}_i \end{aligned} $$ $$ \Updownarrow $$ $$ \begin{aligned} &\sum_{d_{-i}, z_i} \Big[ \overbrace{(\pi(s_i(\text{I}_i), d_{-i}, z_i)- \pi(d, d_{-i}, z_i))}^{\equiv\Delta \pi(s_i(\text{I}_i), d, d_{-i}, z_i)}\\ & \times \mathbb{P}(D_{-i}=d_{-i}, Z_i=z_i| \text{I}_i)\Big]\geq 0 \text{ }\forall d \in \mathcal{D}_i\text{, }\forall s_i\text{, } \text{ a.s.} \text{I}_i \end{aligned} $$
(3) Multiplying by $\mathbb{P}(D_i=s_i(\text{I}_i)| \text{I}_i)>0$, above becomes $$ \begin{aligned} &\sum_{d_{-i}, z_i} \Big[ \Delta \pi(s_i(\text{I}_i), d, d_{-i}, z_i) \times \mathbb{P}(D_{-i}=d_{-i}, Z_i=z_i| \text{I}_i) \Big] \\& \times \mathbb{P}(D_i=s_i(\text{I}_i)| \text{I}_i) \geq 0 \text{ }\forall d \in \mathcal{D}_i\text{, }\forall s_i\text{, }\text{ a.s.} \text{I}_i \end{aligned} $$
(4) Summing over $s_i$, above implies $$ \begin{aligned} &\sum_{s_i, d_{-i}, z_i} \Big[ \Delta \pi(s_i(\text{I}_i), d, d_{-i}, z_i) \times \mathbb{P}(D_{-i}=d_{-i}, Z_i=z_i| \text{I}_i) \\ & \times \mathbb{P}(D_i=s_i(\text{I}_i)| \text{I}_i) \Big] \geq 0 \text{ }\forall d \in \mathcal{D}_i\text{, }\text{ a.s. }\text{I}_i \end{aligned} $$
(5) By conditional independence, above becomes $$ \begin{aligned} &\sum_{s_i, d_{-i}, z_i} \Big[ \Delta \pi(s_i(\text{I}_i), d, d_{-i}, z_i) \\ &\times \mathbb{P}(D_i=s_i(\text{I}_i), D_{-i}=d_{-i}, Z_i=z_i| \text{I}_i) \Big] \geq 0 \text{ }\forall d \in \mathcal{D}_i\text{, } \text{ a.s. } \text{I}_i \end{aligned} $$ $$ \Updownarrow $$ $$ \begin{aligned} &E_{D_i, D_{-i}, Z_i} (\Delta \pi(D_i, d, D_{-i}, Z_i) | \text{I}_i) \geq 0 \text{ }\forall d \in \mathcal{D}_i\text{, }\text{ a.s. } \text{I}_i \end{aligned} $$
Of this proof what I'm not convinced about is the first claim: If $s_i$ denotes PSBNE played by player $i$, then $$ \begin{aligned} &E_{D_{-i},Z_i}\Big( \pi(s_i(\text{I}_i), D_{-i}, Z_i)\Big| \text{I}_i, s_i \Big) \\ &\geq \max_{d\in \mathcal{D}_i} E_{D_{-i},Z_i}\Big( \pi(D_i, D_{-i}, Z_i)\Big| \text{I}_i, D_i=d\Big) \text{ } \forall s_i\text{, } \text{ a.s. }\text{I}_i \end{aligned} $$ This is not the definition of PSBNE, where we also condition on $s_{-i}$, and it does not even seem necessary for it! In other words $$E_{D_{-i},Z_i}\Big( \pi(s_i(\text{I}_i), D_{-i}, Z_i)\Big| \text{I}_i, s_i \Big)\neq E_{I_{-i},Z_i}\Big( \pi(s_i(\text{I}_i), s_{-i}(I_{-i}), Z_i)\Big| \text{I}_i, s_i, s_{-i} \Big) $$