Suppose I have a game on $n$ players and a sequence of strategy profiles $(s_1^{(1)},\dots,s_n^{(1)}), (s_1^{(2)},\dots,s_n^{(2)}), (s_1^{(3)},\dots,s_n^{(3)}), \dots$.
Each $(s_1^{(i)},\dots,s_n^{(i)})$ is a $\epsilon_i$-Nash equilibrium, and the sequence $\epsilon_1,\epsilon_2,\epsilon_3,\dots$ converges to zero.
My questions:
When (under what conditions/assumptions) do all the strategies converge? That is, for each player $j$, $s_j^{(1)},s_j^{(2)},s_j^{(3)},\dots$ necessarily converges.
Under what further conditions is the limit of this sequence actually a Nash equilibrium of the game?
Thanks very much!
This was too long for a comment. Do you have a specific game being considered? Just your property 1 is not true for in general, even for $2 \times 2$ games.
For convergence, you need that, in two strategy profiles, if player $i$ is best-responding up to $\epsilon_1$ and $\epsilon_2$ respectively, then the distance (assuming strategies lie in a metric space) between his strategies $s_1$ and $s_2$ is controllable by $|\epsilon_1 - \epsilon_2|$. Take any $2 \times 2$ game with two Nash equilibrium strategy profiles $\sigma_1$ and $\sigma_2$. Let player 1's strategies in the two profiles be $s_1$ and $s_2$ respectively and $s_1 \neq s_2$. Here $\epsilon_1 = \epsilon_2 = 0$. But the sequence of strategies $s_1, s_2, s_1, s_2, \cdots$ is clearly divergent.
For (2), again in general it's too much to hope for. Let's assume $s_i^1 \rightarrow s_1$ with $\epsilon_i \rightarrow 0$. The sequence $\{s_i^1\}$ is only one way to perturb the response $s_1$. This is much weaker than the definition of NE. (What you have is something akin to the consistency condition for off-equilibirum beliefs in a sequential equilibrium. Stability under one sequence of approximating beliefs is much weaker than stability under any small perturbation of the given belief.)
But in specific models you can arrange for these (very special) conditions. For example, in the Rubinstein bargaining model, this is probably true.