Wednesday, January 11, 2012

A Curious Croquet Game

On January 7, 2012 I posted a discussion of Weinberger's article entitled "To Know, But Not Understand."

I thought of this example of a complex system that you can understand but not know. You can understand the system and all its elements but not know what it's next state will be.

From Alice’s Adventures in Wonderland by Lewis Carroll

"Get to your places!" shouted the Queen in a voice of thunder, and people began running about in all directions, tumbling up against each other: however, they got settled down in a minute or two, and the game began.

Alice thought she had never seen such a curious croquet ground in her life: it was all ridges and furrows: the croquet balls were live hedgehogs, and the mallets live flamingos, and the soldiers had to double themselves up and stand on their hands and feet, to make the arches.

The chief difficulty Alice found at first was in managing her flamingo: she succeeded in getting its body tucked away, comfortably enough, under her arm, with its legs hanging down, but generally, just as she had got its neck nicely straightened out, and was going to give the hedgehog a blow with its head, it would twist itself round and look up in her face, with such a puzzled expression that she could not help bursting out laughing; and, when she had got its head down, and was going to begin again, it was very provoking to find that the hedgehog had unrolled itself, and was in the act of crawling away: besides all this, there was generally a ridge or a furrow in the way wherever she wanted to send the hedgehog to, and, as the doubled-up soldiers were always getting up and walking off to other parts of the ground, Alice soon came to the conclusion that it was a very difficult game indeed.

The players all played at once, without waiting for turns, quarreling all the while, and fighting for the hedgehogs; and in a very short time the Queen was in a furious passion, and went stamping about, and shouting "Off with his head!" or "Off with her head!" about once in a minute. Alice began to feel very uneasy: to be sure, she had not as yet had any dispute with the Queen, but she knew that it might happen any minute, "and then," thought she, "what would become of me? They're dreadfully fond of beheading people here: the great wonder is, that there's any one left alive!"

Saturday, January 7, 2012

To Know, but Not Understand: David Weinberger on Science and Big Data

In Atlantic Monthly, January 3, 2012, David Weinberger discusses his new book, Too Big to Know. He has some important insights on complexity to share. He begins the article with:

“Thomas Jefferson and George Washington recorded daily weather observations, but they didn't record them hourly or by the minute. Not only did they have other things to do, such data didn't seem useful. Even after the invention of the telegraph enabled the centralization of weather data, the 150 volunteers who received weather instruments from the Smithsonian Institution in 1849 still reported only once a day. Now there is a literally immeasurable, continuous stream of climate data from satellites circling the earth, buoys bobbing in the ocean, and Wi-Fi-enabled sensors in the rain forest. We are measuring temperatures, rainfall, wind speeds, C02 levels, and pressure pulses of solar wind. All this data and much, much more became worth recording once we could record it, once we could process it with computers, and once we could connect the data streams and the data processors with a network.”

What he is writing about here is the concept of scale in complexity, although he does not use that term. There are many systems in nature, and those made by man, that appear simple or complicated on one scale, but complex on another. The coastline is one of those. Depending on the scale, the coast line can appear either, simple or complicated, and on a fine scale it appears complex, an infinite fractal. As someone once answered the question, “How long is our coast line?” with, “It depends upon the size of the ruler.” Markets exhibit this phenomena as well, especially a stock market. Brokers like to show prospective buyers the simple view – long term with a trend towards increasing value. But, as the data is examined in a finer time scale, the behavior of the stock market becomes complex and unpredictable.

Weinberger is getting to the heart of the issue of the application of complexity science to the affairs of humans. People back away from complexity because they don’t know what to do with it. Of what value is the concept of complexity when all it tells you is that you can’t predict the behavior of the systems they have to work with, or in the long run predict the future.

He suggests that we need a new definition of knowing. “How will we ever make sense of scientific topics that are too big to know? The short answer: by transforming what it means to know something scientifically.”

In my words, he is suggesting a type of meta knowledge that still has enormous value.

“The problem -- or at least the change -- is that we humans cannot understand systems even as complex as that of a simple cell. It's not that were awaiting some elegant theory that will snap all the details into place. The theory is well established already: Cellular systems consist of a set of detailed interactions that can be thought of as signals and responses. But those interactions surpass in quantity and complexity the human brains ability to comprehend them. The science of such systems requires computers to store all the details and to see how they interact. Systems biologists build computer models that replicate in software what happens when the millions of pieces interact. It's a bit like predicting the weather, but with far more dependency on particular events and fewer general principles.

Models this complex -- whether of cellular biology, the weather, the economy, even highway traffic -- often fail us, because the world is more complex than our models can capture. But sometimes they can predict accurately how the system will behave. At their most complex these are sciences of emergence and complexity, studying properties of systems that cannot be seen by looking only at the parts, and cannot be well predicted except by looking at what happens.

This marks quite a turn in science's path. For Sir Francis Bacon 400 years ago, for Darwin 150 years ago, for Bernard Forscher 50 years ago, the aim of science was to construct theories that are both supported by and explain the facts. Facts are about particular things, whereas knowledge (it was thought) should be of universals. Every advance of knowledge of universals brought us closer to fulfilling the destiny our Creator set for us.”

The problem I see with what he presents in this article is that there are many complex systems that do not lend themselves to this type of modeling. No amount of data will change that. In fact, the more data, the worse things get. And, there are systems whose models are so dependent on initial conditions that there is no reliability in the results, as Lorenz discovered in his modeling of the weather.

So, at this point, I’m not convinced by his examples, but I am by his idea that we have to develop a new concept of what it means to know a system.

I’ve ordered the book, and I’m hoping that the answer may be there.

Right now I think that I can understand a complex system, but not be able to know how it is going to behave.

Too Big to Know: Rethinking Knowledge Now That the Facts Aren't the Facts, Experts Are Everywhere, and the Smartest Person in the Room Is the Room, David Weinberger, Basic Books, 2012