Modelling the Future  

The Edge has an interesting column by Stephen Schneider (of RealClimate, amongst other things) on modelling complex systems. Its way too long to quote all, or even most, of it (yes - even I have limits) so go and read the rest of it at the link.

There's some debate as to the predictive validity of modeling between experimentalists and theoreticians or, in the case of climate, modelers and observationalists. I think it's a false debate. You can't do one without the other.

First of all, there are no observations of the future until it happens and by then it's too late. Therefore the future is by definition a model. Now: the model itself is our best understanding of how the atmosphere works, how the oceans work, how ecosystems work, how chemical exchanges work. There are all kinds of models: models of social components that try to predict population growth, demographic models that predict population sizes, standards of living predictions from economic models, political models, technological and scientific models, etc. etc.

Every one of those is based upon empirically grounded sciences that look at the current and the past of different experiments or societies, and then construct a crystal ball that makes a prediction. The crystal ball is kind of cloudy, but does allow one to project forward alternative futures under different assumptions of human behavior that modifies the atmosphere. To model successfully, we try to assess confidence based upon how much we believe their structural integrity will be viable over time.

I'll give you two radically different examples. Somebody predicts an eclipse 28 years from now and they do it to with an uncertainty of a matter of minutes. This prediction is based on a model and most of us will bet a fair amount of money that they will get it almost right. It's possible that an unpredicted asteroid will go by and somehow disturb gravity, and screw it up a bit and throw off the prediction slightly. Why is it that that prediction, even though it's a counter-factual, is deemed reliable?

Because we have such underlying confidence in the equations of celestial dynamics and so forth that we assign a very high confidence within a narrow range. Even though there's always a probability function and there's always some uncertainty that our median, best guess or mode, isn't quite right, the tails of the probability distribution are really tight.

Now let's look at a different type of model, one that tries to predict how people will value species of birds versus the development rate of the economy in a bird-rich tropical country. You can go to that country now and you can take a look at their values. But, in order to know what they're values are going to be in a hundred years, you have to figure out what the threats are to the birds, as well as how that culture is going to change. To do this, you have to make deep underlying assumptions about both nature of species viability and a culture on which you really have only limited empirical data.

You're going to make a prediction because you have to and because it's a relevant question, but your probability function may be so broad that you have little real quantitative information other than the fact that anything from little effect to catastrophe are still probable at more than a few percent chance. Assessments of both potential outcomes and associated probabilities are based upon both mental and mathematical models, which are your best codification of our current understanding, and are to the extent possible empirically grounded.

But the not-empirical question is, how good will the assumptions that you used to construct those models be in the future? Your confidence is going to depend upon how much you believe that the structural integrity of models built from today's conditions will hold in a very different future world.

Think about ladies' hats and Easter. Back in the Easter parades down Fifth Avenue in the Victorian era, it was fashionable to have 30 cedar waxwing bird carcasses across the top of the hat. I don't even think you would find the most rabid anti-environmentalist doing that any more! We've had a radical cultural change that we don't want to ravage nature for fashion. Was that predictable in 1900? I'm not sure. We always have to be relatively non-arrogant when we're talking about predicting social change, evolving value preferences and technological change. Yet predict it we must, if we want to have some idea about what the space is within which we're operating over time.

I keep arguing, don't be too arrogant about the belief in your models; what you do is make projections, and then you crank a knob to try to avoid the more catastrophic outcomes or the outcomes that don't match your values, but we better be reflexive. That means we had better have what in the language of the systems guys is a "complex adaptive system". We need to always build in knobs so that as we get new information that changes our understanding of the structural bases of models, we can in turn crank up or down our degrees of policy control. But we rationalists, we systems analysts, think that's a great model of reality.

But then I go to talk with members of Congress, and I talk to government ministers in other countries. I discover, often over a good glass of wine, that they think that's a rational measure of good policy, but that they are nonetheless going to go through a five-year knock-down political debate—with the auto industry that wants to make big fat polluting cars, and the mining industry that wants to get the coal and uranium at lowest cost—and when all is said and done, they will reach some political compromise that satisfies nobody. But at least it's progress—and the last thing anybody wants to do is go back in five years and revisit that ugly debate and rekindle, among all these already not very satisfied stake-holders, more reasons for them to be angry.

That is why when we intellectuals say, hey, set up a complex-adaptive system, adjust the management knobs as we learn more; politicians say, oh no, we solved it, don't reopen festering compromises. Problem's over. Don't take me back there anytime soon.

As scientists, if we think we're going to make a difference in the world, we have to get beyond the rationality of our own inside-the-club thinking in which adjusting knobs and building complex-adaptive systems are the way to go. We have to think about the public as well as the various stakeholders and decision-makers, and figure out how we can make it safe for the honest politician—not always an oxymoron—to be able to build a rational management system that recognizes that very wide tails on uncertainty distribution mean that you have to be able to crank up or down as you learn more; that you can't claim that the problem's solved and never revisit an issue once legislation is passed. But we also need to put ourselves in the shoes of those making policy and being buffeted by it as part of our design process.