I am trying to understand how the replicator dynamic can be derived as a gradient flow of the Fisher information metric (aka Shahshahani metric). I have a question about understanding a particular proof (which may be answerable without particular knowledge about the replicator dynamic), but if you have a different straightforward derivation/proof that would also be very helpful.
I am trying to understand the following proof (I'll ask my particular questions at the end):
Consider the replicator dynamic $\dot{p}_i = p_i(f_i(p) - \bar{f})$, where $f_i(p) = \frac{ \partial V}{\partial p_i}$ for $i \in 1, \dots, m$, and $\bar{f} = \sum_i p_i f_i(p)$
Lemma: $$ \dot{p}_i = (\nabla_g V)_i = \sum_j g_{ij} \frac{\partial V}{\partial p_j} $$ where $g_{ij}$ is the inverse of the Fisher information metric.
Proof: For every $\xi = (\xi_1, \dots, \xi_k)$ with $\sum_j \xi_j = 0$, that is, for every tangent vector to the simplex in which the dynamics of $p$ take place, we have: \begin{align} \sum_j g_{ij} \dot{p}_i \xi_j &= \sum_j \frac{1}{p_j} \dot{p}_j \xi_j \\ &= \sum_j (f_j(p) - \bar{f}) \xi_j \\ &= \sum_j f_j(p) \xi_j \\ &= \sum_j \frac{\partial V}{\partial p_j} \xi_j \\ &\text{which implies the lemma} \end{align}
My questions are:
- I am not sure why they start the proof how they do. My understanding is they start the proof with the expression of the dot product between the vector pointing in the direction of change ($\dot{p}$) and an arbitrary other tangent vector to $p$. Why start with that dot product?
- I don't know how the last step of the proof implies tne Lemma.
Any clarification, exposition, intuition, or alternative proof that can help me fully understand how the replicator dynamic is derived as a gradient flow of the Fisher metric would be extremely appreciated!
This is very old, but I'll try to clarify on the matter for future reference.
Some history
First of all: it is not true in general that the replicator dynamic is a gradient with respect to the Shahshahani metric. What you are trying to prove is that this is true in the special case where the payoff function $f$ is itself a Euclidean gradient with potential function $V$.
The proof you provide can be found in [HS] p. 258. There are a couple of typos in your question, but the statement is the following:
Let the payoff $f$ be a Euclidean gradient with potential function $V$. Then the corresponding replicator field is a Shahshahani gradient with the same potential function.
Let me not go into the details of your proof, since there are (at least) two more modern and geometrical ways to prove this statement.
Def. A potential game is a game whose payoff function is an Euclidean gradient: $f_i = \partial_i V$.
Some history. Citing Sandholm [S]:
Shahshahani (1979), building on the early work of Kimura (1958), showed that the replicator dynamic for a potential game is a gradient dynamic after a “change in geometry,” that is, after the introduction of an appropriate Riemannian metric on int(X) [the Shahshahani metric]. Subsequently, Akin (1979; 1990) proved that Shahshahani’s (1979) result can also be represented using the change of variable presented in theorem 7.1.6. The direct proof offered in the text is from Sandholm, Dokumacı, and Lahkar (2008).
Sandholm, Dokumacı, and Lahkar [S+] as well as Sandholm [S] prove this statement by means of the Akin isometry: the map $H(x) = 2\sqrt{x}$ from the positive orthant to itself is an isometry between the Shahshahani metric and the Euclidean metric. Pushing the replicator field for a potential game along this isometry and studying the resulting system shows that it is indeed a gradient system.
There is though a more general result, that can be found in [MS] and that implies this result as a simple corollary: this is what I'll show here. Some familiarity with differential geometry is required.
The proof
(For whatever reason Tex is not rendering curly brackets, so I will denote sets with square brackets).
A single-population game is given by
This is a crucial point: the payoff field is a 1-form, not a vector field.
Evolutionary dynamics
The question is now, can we model the evolution of the population state in time as a consequence of a process of learning and adaptation, in which the agents in the population are trying to maximize their payoff while learning about the actions of other agents? [F+]
More geometrically, given a population game, how can we get a dynamical system on the simplex $$\dot{x} = X(x)$$ modeling the evolution of the population state? There are plenty of ways of doing so; the replicator dynamics is a milestone one.
Endow the simplex with the Shahshahani metric $$g_{ij}(x) = \frac{\delta_{ij}}{x_i}$$
Def. The replicator vector field $X$ is the orthogonal projection of the dual of the payoff field with respect to the Shahshahani metric: $$X = P\left(f^{\sharp}\right)$$
The sharp operator gives you a vector field from the 1-form $f$; and the orthogonal projection operator $P$ projects this vector field on the simplex. Note that both $\sharp$ and $P$ are Riemannian objects, i.e. they depend on the metric.
Let's see that this gives the usual replicator field. In components, the sharp operator acts by matrix multiplication by the inverse metric, denoted by $$g^{ij}(x) = \delta_{ij} x_i$$ $$(f^{\sharp})^i(x) = \sum_j g^{ij}f_j = \sum_j \delta_{ij}x_if_j = x_i f_i$$
Some calculus will show you that the orthogonal projection of a vector $X$ on a surface $S$ with respect to a metric $g$ is $$P(X) = X - \frac{g(X,n)}{g(n,n)}n$$ where $n$ is any vector normal to $S$ with respect to $g$.
Let $\mathbb{1}$ be the 1-form with $1$ in each component, and denote by $\cdot$ the action between a dual vector and a vector. In our case the surface $S$ is the simplex with normal vector $n = \mathbb{1}^{\sharp}$, so $$ \begin{split} P(f^{\sharp}) & = f^{\sharp} - \frac{g(f^{\sharp},n)}{g(n,n)}n \\ & = f^{\sharp} - \frac{f^{\sharp}\cdot \mathbb{1}}{n \cdot \mathbb{1}}n \\ \end{split} $$ Note that $$n^i(x) = (\mathbb{1}^{\sharp})^i(x) = \sum_j g^{ij} = \sum_j\delta_{ij}x_j = x_i$$ Evaluating $P\left(f^{\sharp}\right)$ at a point $x$ on the simplex then yields $$\left(P\left(f^{\sharp}\right)\right)^i(x) = x_if_i - \frac{\sum_j x_jf_j}{\sum_hx_h}x_i = x_i\left(f_i(x)-\bar{f}(x)\right)$$ q.e.d.
Corollary: the potential case
Finally, a potential game has a payoff field that is a Euclidean gradient - more precisely, a payoff field that is an exact 1-form: $$f = \text{d}V$$ It is then immediate to see that in this case the replicator field is itself a gradient (recall that the definition of gradient of a function is $\text{grad}f = (df)^{\sharp}$, which coincides with the usual array of partial derivatives in the case of the Euclidean metric): $$X = P(f^{\sharp}) = P\left((\text{d}V)^{\sharp}\right) = P(\text{grad}V)$$ It's a standard result that the orthogonal projection on a subspace of the gradient of a function is the gradient of the function restricted to the subspace, so in conclusion $$X = \text{grad}\left(V|_{\mathcal{X}}\right)$$
[HS] Hofbauer J, Sigmund K (1998) Evolutionary games and population dynamics (Cambridge University Press).
[S] Sandholm WH (2010) Population games and evolutionary dynamics (MIT press).
[MS] Mertikopoulos P, Sandholm WH (2018) Riemannian game dynamics. Journal of Economic Theory 177:315–364.
[S+] Sandholm WH, Dokumacı E, Lahkar R (2008) The projection dynamic and the replicator dynamic. Games and Economic Behavior 64(2):666–683.
[F+] Fudenberg D, Drew F, Levine DK, Levine DK (1998) The theory of learning in games (MIT press).