For the past month I’ve been embroiled in two independent, vital, strategic decision-making processes. What school should my son go to next, and what might the future of particle physics be? Game theory has a cautionary tale

Choice of school and choice of future experiment both involve many parameters and constraints, and they involve many well-informed, self-interested parties independently trying to obtain the best outcome for themselves. Or for their child, or their cherished sub-field of science.

It becomes clear quite quickly that a big factor in any individual’s decision is: What decision is everyone else making? For example, if you decide to work on a particular new collider but no one else in Europe does, you are very unlikely to succeed. Similarly, if you know that lots of parents are choosing a given school, it is likely to be a school full of well-supported and well-motivated children; i.e. better for your child too.

There’s a general mood in political and public life to the effect that choice is good, and that increased choice is better. This is often bolstered by a vague Darwinian sense of “survival of the fittest”; clearly over the past billion years or so, chaotic choices (actually random mutations combined with external pressures) have led to some pretty wonderful solutions to the problem of how to survive and reproduce.

But quite apart from the carnage of failed mutations along the way, there’s really no sense in which the solutions arrived at in such a process are guaranteed to be optimal. At some point (I can’t remember whether I was being addressed by an earnest head-teacher about the value they place on each individual child, or by a physicist about the timeliness and impact of their proposed project) I found myself thinking about game theory, Nash equilibrium and Braess’ paradox.

Ok, I didn’t; I found myself vaguely remembering a road network I’d seen somewhere where if you build a new road, you slow everybody’s journey down. Then I went to wikipedia and discovered this is known in game theory as Braess’ paradox and is an example of a non-optimal Nash equilibrium.

If you have a bunch of independent decision makers (parents, physicists, drivers), all trying to obtain a given goal (e.g. the quickest journey from A to B), making selfish decisions but taking each others’ decisions into account, a “Nash equilibrium” is a stable situation where everyone knows what everyone else is doing and no-one has anything to gain by unilaterally changing their decision.

The network in Braess’ paradox is set up such that there are two kinds of road; fast roads which slow down as they get more congested, and slow roads where congestion makes no difference. There are two routes from A to B, one which is firstly on a slow road then a fast road, the other which is firstly on a fast road then on a slow road. In Nash equilibrium, it makes no difference which route you take, so people will use both routes equally.

If you build a connecting road in the middle, however, people can choose to use fast roads all the way. But since the fast roads are sensitive to congestion, they slow down. Everyone loses.

You might say “Ha, I’m smart, I will ignore the congested fast roads.” But you have the slow road all the way now, and this is still slower than the congested “fast” roads.

You might also realise the connecting road is the problem, and so choose to ignore it, going half by slow road and half by fast road as before. But again you will be slower than trying the fast roads all the way. And all these routes are slower than if the new connecting road had never been built, because the fast roads are now more congested than before. If you don’t believe me, the maths working is shown here.

The only way to recover the original, faster journey time, is for all drivers to make a collective decision to ignore the new road. This would improve the journey times for everyone. But without enforcement, this is unstable, since any antisocial individual who ignores the boycott would have an even faster journey time, taking uncongested fast roads the whole way, while slowing everyone else marginally. This would continue to be the best choice for individuals until we’re back were we started and everyone is again slower than they would have been if the new road had never been built.

There’s no great revelation here, except to say that it is mathematically proven that collective decision-making can, under some circumstances, achieve hugely better outcomes than a bunch of smart but independent decisions. While I see little chance of achieving this in the case of school choice, it does seem as though it might be doable in particle physics, and the thought has already sustained me through hours of highly variable presentations and discussions in Krakow, Birmingham and Swindon over the past weeks.

At the Guardian.