I'm reading this lecture slides to understand the underlying theories of Fictitious play, but I get stuck when reading Page 21. Here's the snapshot of that page
where $BR_i$ is the best response of player $i$, $p_i$ is the average strategy of player $i$, $p_{-i}$ is the average strategy of players other than $i$, $\Sigma$ is the strategy space, and u_{i} is the utility function which player $i$ aims to maximize.
I don't understand why it uses the inclusion sign $\in$ in CTFP but uses the equality $=$ in perturbed CTFP. I'm aware of the existence of explanation--"Since $C_i$ is uniquely defined, the perturbed CTFP is described by a differential equation rather than a differential inclusion."--but I don't know how to make sense of it. To me, the term differential inclusion is quite strange and I still cannot understand it even after reading the wiki page and several other resources.
The notation is a little bit confusing which might be the source of your problems.
The authors define $$BR_i(p^t_{-i}) = argmax_{\sigma_i \in \Sigma_i} u_i(\sigma_i, p^t_{-i})$$ which is not correct. It should be written as $$BR_i(p^t_{-i}) \in argmax_{\sigma_i \in \Sigma_i} u_i(\sigma_i, p^t_{-i})$$. (After all, given an opponent's strategy, you tend to have multiple strategy that is best response to it.)
In the second one, you have, $C_i(p^t_{-i}) = argmax_{\sigma_i \in \Sigma_i} u_i(\sigma_i, p^t_{-i}) - V_i(\sigma_i)$, which is correct. This is because $-V_i(\sigma_i)$ is strictly concave. $u_i(\sigma_i, p^t_{-i})$ is linear in its argument $\sigma_i$. You are maximizing a strictly concave function. This will always give you a single maximizer.
Does this help?