« Code 46 | Home | 1800GotJunk.com »

March 12, 2005

Why can't statistical theory predict catastrophes?


John Kay is a superb writer on complex things.

He makes them understandable.

By definition, anyone who can do that is a superb writer.

Most people, dealing with difficult subjects, make a hash of it, rendering the reader perplexed, angry at her own apparent stupidity, and vowing never again to read about said topic.

The column that follows is from The Financial Times; it appeared there earlier this year, but that doesn't matter: it would have as much applicability five years from now, or even 50.

The subject is why it's so darned difficult to anticipate extreme, catastrophic events, even with the armamentarium of modern day statistics and computer modeling.

Read his piece, you'll understand the subject much better than you did.

Full disclosure: I am not an employee of John Kay, nor have I ever received any consideration, financial or otherwise, from him.

I have corresponded with him via email, and I did, of my own volition, purchase his latest book,


"Everlasting Light Bulbs: How Economics Illuminates the World."

Here's the column.

    The odds of finding a formula to foretell disaster

    I first heard the word tsunami from a colleague interested in the mathematics of extreme statistics. He was wondering why there are more catastrophes in the world than there should be.

    At first sight, that seems a silly question. One disaster such as the Indian Ocean tidal waves is one too many. But those who study extreme statistics ask why there are more catastrophic events than are consistent with standard theories of how the world works.

    Our expectations of events that are hard to predict - earthquakes, financial crises, political upheavals - are founded on classical theories of probability and statistics developed in the 18th and 19th centuries. These use simple mathematical models to make generalisations about the world.

    At the University of Strasbourg, Professor Ladislaus Bortkiewicz, who knew nothing about horses or military training, predicted the incidence of deaths from horse kicks among Prussian officer cadets. A Guinness employee worked out the same formula to explain the fermentation of stout. And these same equations were indispensable when telephone engineers calculated how many exchange lines to install.


    The sense of wonderment I felt on learning the range of these regularities has never quite left me. The distribution of household incomes has similar mathematical properties to the distribution of the length of human life. And Myron Scholes won the Nobel Prize in economics for discovering that the equations needed to value complex derivative securities also describe how bath salts disperse in the tub.

    The scope and achievements of these theories are so great that it is a shock when they do not work. The stock market rarely moves by more than 1 or 2 per cent in a day. Classical statistical theory therefore tells us that Black Monday, October 19, 1987, when stock markets fell by more than 20 per cent, could not have happened. In terms of probability, it was like simultaneously being struck by lightning, hit by a meteorite and felled by a stray rifle shot. Similarly, there are too many large earthquakes, given the frequency of smaller earthquakes.

    So different models are needed. In classical statistical distributions, compounding events are independent of each other. Professor Bortkiewicz's theory worked because undisciplined horses were rare in the Prussian army. Householders make independent decisions to pick up the phone, and individually unpredictable decisions become predictable in aggregate. But if mad horse disease had spread across Prussia, Bortkiewicz would never have secured an appointment in Berlin. Phone lines jam when everyone wants to make the same inquiry, tell the same news, or share the same greeting.

    And such a cascade of events led to the Aceh earthquake and the Indian Ocean waves. Rocks are constantly in motion along the fault lines between tectonic plates. Every movement provokes a movement in every surrounding rock, which transmits the disturbance. So shocks may be amplified many times: sometimes, as on December 26, to devastating effect.

    Cascade models also have uses outside physics. Many economists describe a world of rational, independent decision makers, eagerly assimilating each new piece of information so that at every moment markets reflect the distillation of all available knowledge. But business and finance is more like the fault systems in the oceans, in which vibrating rocks transmit their instability to each other: greed, fear, panic and misinformation are spread like measles, or forest fires, and the same models that describe measles, forest fires and earthquakes apply. Contagious processes give rise to the Black Monday crash and the 3G auction fiasco, the new economy boom and the dollar's rollercoaster ride.

    It is perhaps easier for Indonesian villagers than for western rationalists to accept that big events have no real cause, that small slippages can become catastrophes that we can neither predict nor prevent. Still we can calculate their frequency, watch them developing and forecast where they are most likely to happen. The lesson from the Indian Ocean disaster is that there is nothing you can do to control extreme statistics but a lot you can do to insulate yourself from the consequences. That lesson is as relevant to American companies as to Indonesian fishermen.

March 12, 2005 at 09:01 AM | Permalink


TrackBack URL for this entry:

Listed below are links to weblogs that reference Why can't statistical theory predict catastrophes?:


The comments to this entry are closed.