Chapter 11 - Population Games and Evolutionary stable strategies

Recap

We considered infinitely repeated games using a discount rate;
We proved a powerful result stating that for a high enough discount rate player would cooperate.

In this chapter we’ll start looking at a fascinating area of game theory.

Population Games

In this chapter (and the next) we will be looking at an area of game theory that looks at the evolution of strategic behaviour in a population.

Definition of a population vector

Considering an infinite population of individuals each of which represents a strategy from $\Delta S$, we define the population profile as a vector $\chi\in[0,1]^{|S|}_\mathbb{R}$. Note that:

$\sum_{s\in S}\chi(s)=1$

It is important to note that $\chi$ does not necessarily correspond to any strategy adopted by particular any individual.

Example

Consider a population with $S=\{s_1,s_2\}$. If we assume that every individual plays $\sigma=(.25,.75)$ then $\chi=\sigma$. However if we assume that .25 of the population play $\sigma_1=(1,0)$ and .75 play $\sigma_2=(0,1)$ then $\chi=\sigma$.

In evolutionary game theory we must consider the utility of a particular strategy when played in a particular population profile denoted by $u(s,\chi)$ for $s\in S$.

Thus the utility to a player playing $\sigma\in\Delta S$ in a population $\chi$:

$u(\sigma,\chi)=\sum_{s\in S}\sigma(s)u(s,\chi)$

The interpretation of the above is that these payoffs represent the number of descendants that each type of individual has.

Example

If we consider a population of N individuals in which $S=\{s_1,s_2\}$. Assume that .5 of the population use each strategy so that $\chi=(.5,.5)$ and assume that for the current population profile we have:

$u(s_1,\chi)=3\text{ and }u(s_2,\chi)=7$

In the next generation we will have $3N/2$ individuals using $s_1$ and $7N/2$ using $s_2$ so that the strategy profile of the next generation will be $(.3,.7)$.

We are going to work towards understanding the evolutionary dynamics of given populations. If we consider $\chi$ to be the startegy profile where all members of the population play $\sigma^*$ then a population will be stable only if:

$\sigma^*\in\text{argmax}_{\sigma\in\Delta S}u(\sigma,\chi)$

Ie at equilibrium $\sigma^*$ must be a best response to the population profile it generates.

Theorem for necessity of stability

In a population game, consider $\sigma^*\in\Delta S$ and the population profile $\chi$ generated by $\sigma^*$. If the population is stable then:

$u(s,\chi)=u(\sigma^*,\chi)\text{ for all }s\in\mathcal{S}(\sigma^*)$

(Recall that $\mathcal{S}(s)$ denotes the support of $s$.)

Proof

If $|\mathcal{S}(\sigma^*)|=1$ then the proof is trivial.

We assume that $|\mathcal{S}(\sigma^*)|>1$. Let us assume that the theorem is not true so that there exists $\bar s\in\mathcal{S}(\sigma^*)$ such that:

$u(\sigma^*,\chi)\ne u(\bar s,\chi)$

Without loss of generality let us assume that:

$\bar s=\text{argmax}_{s\in\mathcal{S}(\sigma^*)}u(s,\chi)$

Thus we have:

$\begin{aligned} u(\sigma^*,\chi)&=\sum_{s\in\mathcal{S}(\sigma^*)}\sigma^*(s)u(s,\chi)\\ &\leq\sum_{s\in\mathcal{S}(\sigma^*)}\sigma^*(s)u(\bar s,\chi) \\ &\leq u(\bar s,\chi)\sum_{s\in\mathcal{S}(\sigma^*)}\sigma^*(s) \\ &\leq u(\bar s,\chi) \end{aligned}$

Which gives:

$u(\sigma^*,\chi)< u(\bar s,\chi)$

which implies that the population is not stable.

Evolutionary Stable Strategies

Definition of an evolutionary stable strategy

Consider a population where all individuals initially play $\sigma^*$. If we assume that a small proportion $\epsilon$ start playing $\sigma$. The new population is called the post entry population and will be denoted by $\chi_{\epsilon}$.

Example

Consider a population with $S=\{s_1,s_2\}$ and initial population profile $\chi=(1/2,1/2)$. If we assume that $\sigma=(1/3,2/3)$ is introduced in to the population then:

$\begin{aligned} \chi_{\epsilon}&=(1-\epsilon)(1/2,1/2)+\epsilon\sigma\\ &=(1/2-\epsilon/6,1/2+\epsilon/6) \end{aligned}$

Definition

A strategy $\sigma^*\in\Delta S$ is called an Evolutionary Stable Strategy if there exists an $0<\bar\epsilon<1$ such that for every $0<\epsilon<\bar \epsilon$ and every $\sigma\ne \sigma^*$:

$u(\sigma^*,\chi_\epsilon)>u(\sigma,\chi_\epsilon)$

Before we go any further we must consider two types of population games:

Games against the field. In this setting we assume that no individual has a specific adversary but has a utility that depends on what the rest of the individuals are doing.
Pairwise contest games. In this setting we assume that every individual is continiously assigned to play against another individual.

The differences between these two types of population games will hopefully become clear.

Games against the field

We will now consider an example of a game against the field. The main difficulty with these games is that the utility $u(s,\chi)$ is not generally linear in $\chi$.

Let us consider the Male to Female ratio in a population, and make the following assumptions:

The proportion of Males in the population is $\alpha$ and the proportion of females is $1-\alpha$;
Each female has a single mate and has $K$ offspring;
Males have on average $(1-\alpha)/\alpha$ mates;
Females are solely responsible for the sex of the offspring.

We assume that $S=\{M,F\}$ so that females can either only produce Males or only produce Females. Thus, a general mixed strategy $\sigma=(\omega,1-\omega)$ produces a population with a proportion of $\omega$ males. Furthermore we can write $\chi=(\alpha,1-\alpha)$.

The females are the decision makers so that we consider them as the individuals in our population. The immediate offspring of the females $K$ is constant and so cannot be used as a utility. We use the second generation offspring:

$\begin{aligned} u(M,\chi)&=K^2\frac{1-\alpha}{\alpha}\\ u(F,\chi)&=K^2 \end{aligned}$

Thus:

$u(\sigma,\chi)=K^2\left(\omega\frac{(1-\alpha)}{\alpha}+(1-\omega)\right)$

Note that we can assume that $K=1$ for the rest of this exercise as we will ultimately be comparing different values of the utility.

Let us try and find an ESS for this game:

If $\alpha\ne 1/2$ then $u(M,\chi)\ne u(F,\chi)$ so that any mixed strategy $\sigma$ with support ${M,F}$ would not give a stable population.
Thus we need to check if $\sigma^*=(1/2,1/2)$ is an ESS (this is the only candidate).

We consider some mutation $\sigma=(p,1-p)$:

$\chi_{\epsilon}=(1-\epsilon)\sigma^*+\epsilon\sigma$

which implies that the proportion of population playing $M$ in $\chi_{\epsilon}$ is given by:

$\alpha_{\epsilon}=(1-\epsilon)1/2+p\epsilon=1/2+\epsilon(p-1/2)$

We have:

$u(\sigma^*,\chi_{\epsilon})=1/2+\frac{1-\alpha_{\epsilon}}{2\alpha_{\epsilon}}$

and:

$u(\sigma,\chi_{\epsilon})=(1-p)+p\frac{1-\alpha_{\epsilon}}{\alpha_{\epsilon}}$

The difference:

$\begin{aligned} u(\sigma^*,\chi_{\epsilon})-u(\sigma,\chi_{\epsilon})&=p-1/2+(1/2-p)\frac{1-\alpha_{\epsilon}}{\alpha_{\epsilon}}\\ &=(1/2-p)\frac{1-2\alpha_{\epsilon}}{\alpha_{\epsilon}} \end{aligned}$

We note that if $p<1/2$ then $\alpha_{\epsilon}<1/2$ which implies that $u(\sigma^*,\chi_{\epsilon})-u(\sigma,\chi_{\epsilon})>0$. Similarly if $p>1/2$ then $\alpha_{\epsilon}>1/2$ which implies that $u(\sigma^*,\chi_{\epsilon})-u(\sigma,\chi_{\epsilon})>0$. Thus $\sigma^*=(1/2,1/2)$ is a ESS.