Making Things Work: Solving Complex Problems in a Complex World

Yaneer Bar-Yam writes in the book’s Preface: “In recent years the rapidly changing world around us has been raising concerns about the ability of people to cope with change. Future Shock, The Ingenuity Gap, and other books describe the difficulty of people living in our complex world. Complexity may seem overwhelming but it is not a bad thing. The complexity of the world is a mirror reflection of ourselves working together to make the world work. We, together, are becoming increasingly complex. The reason we can do this is that we work together in increasingly effective ways. We are connected to each other in ways that allow us to respond as teams and organizations. This enables us to do things we would not be able to do by ourselves, not just in terms of amount of effort but in terms of complexity. Complex tasks require complex organizations. When we are part of a complex team we find the world a remarkably comfortable place, because we can act effectively while being protected from the complexity of the world. This feeling is like the experience of a cell in a body, protected from the environment, and contributing to the organism function. Today civilization is the organism we are part of. We are in the midst of a remarkable transition from the individual to the group, organization, and even to global civilization as a functioning unit. While this is a mind bending transition, it is a transition of opportunity for creating a world that works for everybody, on the global level and on the level of each individual.”

Later, he continues this train of thought: “Today we often describe the world around us as highly complex. Complexity manifests in everything from individual relationships to corporate challenges to concerns about the human condition and global welfare. As a global community, we are in the middle of a transition from the industrial to the information age, and this transformation is reflected and rereflected in everything around us. The amount of information that is flowing and the rate of change of society are both aspects of the growing complexity of our existence. As individuals, we have a hard time coping with all the information and change. In some sense more importantly, our society is also having difficulty coping with its own changes.

Our economic and social institutions, that we rely upon at critical times of our lives, including the health and education systems, are changing, not always gracefully, to meet the new challenges. Professional activities, from corporate management to systems engineering, require new approaches, insights and skills. Global concerns, such as environmental destruction and poverty - in developed and undeveloped nations - are becoming more pressing as these changes take place.

Despite major efforts to identify the solutions to these problems, they are often obscure and hidden from us. Even when we think we are making progress, the solutions we think of today may cause us more problems tomorrow. This is because complex problems do not lend themselves to easy solutions. Any action may have hidden effects that cause matters to become worse and the whole strategy we are using may be moving things in the wrong direction. Complex problems are the problems that persist-the problems that bounce back and continue to haunt us. People often go through a series of stages in dealing with such problems-from believing they are beyond hope, to galvanizing collective efforts of many people and dollars to address the problem, to despair, retreat, and rationalization. The progress made seems miniscule compared to the effort and resources expended. Even with all of the modern technological advances, it is easy to become pessimistic about the world today. There is hope, however, in the recognition that people can solve very complex problems when they work together effectively. Unfortunately, this is generally not how we respond when there are problems. We don't always realize the ability that we have when we work together. We tend to assign blame or responsibility to one individual.”

The author summarizes book: “Developing the ability to use a complex systems perspective requires new patterns of thinking. In the first section of this book some of the key complex systems ideas are described. These ideas -like emergence and interdependence-have to do with relationships between parts of a system and how these relationships lead to the behavior of the system. After all, society works because of how people interact with and relate to each other, not how each person acts separately. The results of the interactions between people are patterns of behavior. We will look at how patterns can arise from interactions without someone putting the parts of the pattern in place by telling each person what to do. Using our understanding of how neurons interact in the brain, we will show how the pattern of behavior can be made to serve a purpose. We will find that the type of pattern that arises can be related to how the system is organized - who can interact with whom. We will look more generally at the set of things a system can do, and how this set of actions is related to how it is organized. Some organizations are good at doing complex tasks, and some are not. Perhaps not surprisingly, centrally controlled or hierarchical organizations are not capable of highly complex tasks. This means that we have to figure out how to make distributed/networked organizations if we want to solve complex problems. Finally, we learn about evolution, how really complex systems (including distributed/networked organizations) can form and be effective without being planned (which is crucial because planning them doesn't work!). Counter to how evolution is usually discussed, it is not just about competition, it is always about both competition and cooperation. Competition and cooperation work together at different levels of organization, just as in team sports where players learn to cooperate because of team competition. Making an effective organization is making a successful team.”

He applies these principles to the following systems as examples:

• Health care/medical system
• Education system
• Corporate management
• International development
• Military
• Engineering
• International terrorism

My understanding of his work, acquired only from reading this book, leads me to believe that what he is talking about are complicated or unorganized complex systems. I don’t believe that his approaches will work well for many structured complex systems. He mentions these only once and seems to dismiss their difficulties without explaining how. “Before we can explain how system problems arise and can be fixed, we have to understand something about how systems work. This is where science can help. For many years there has been a sense that chaos and complexity, promising new areas of scientific inquiry, have something fundamental to tell us about the world in which we live. James Gleick's classic book Chaos: Making a New Science (1987) and many other books in later years have raised popular awareness of these directions of research. Much of the focus has been on recognizing the intrinsic unpredictability of nature, and-by extension-of society. However, beyond the fascinating applications to turbulence, meteorology, and other complex problems in the natural world, complex systems science has more to tell us about the world-including human beings and their interactions-than just that it is unpredictable.”

There are two concepts in this book that I have found very helpful to my thinking:

1. You need a complex system to solve a complex problem. I had intuited this earlier (Simple, Complicated or Complex), but he draws the implications out even more than I have. Humans are complex systems. Groups of humans and technology working together are an even more complex system. Therefore you can’t use a simple or a complex system to solve the problems of these types of systems. Even a single human as complex as he or she is, can’t grasp these types of systems. The only way we can solve the problems of these types of systems are with collaborative, creative groups of humans and technology. (See 1, 2 a Few and Many for descriptions of the types of complexity)
2. Systems can appear simple, complicated or complex at different scales.

Bar-Yam summarizes his book this way: “To solve complex problems we must create effective complex organizations. The underlying challenge of this book is the question: How do we create organizations that are capable of being more complex than a single individual? Living with complexity is challenging, but we can and should clearly understand the nature of how it can be done, both for individuals and organizations. The complexity of each individual or organization must match the complexity of the task each is to perform. When we think about a highly complex problem, we are generally thinking about tasks that are more complex than a single individual can understand. Otherwise, complexity is not the main issue in solving it. If a problem is more complex than a single individual, the only way to solve it is to have a group of people-organized appropriately - solve it together. When an organization is highly complex it can only function by making sure that each individual does not have to face the complexity of the task of the organization as a whole. Otherwise failure will occur most of the time. This statement follows quite logically from the recognition of complexity in problems we are facing.

Our experience with organizing people is for large-scale problems that are not very complex. In this case the need for many people arises because many individuals must do the same thing to achieve a large impact. In this old reason for organizing people, a hierarchy works because it is designed to amplify what a single person knows and wants to do. However, hierarchies (and many modifications of them) cannot perform complex tasks or solve complex problems. Breaking up (subdividing) a complex task is not like breaking up a large scale task.

The challenge of solving complex problems thus requires us to understand how to organize people for collective and complex behavior. First, however, we have to give up the idea of centralizing, controlling, coordinating and planning in a conventional way. Such efforts are the first response of almost everybody today because of the effectiveness of this approach in the past. Instead, we need to be able to characterize the problem in order to identify the structure of the organization that can solve it, and then allow the processes of that organization to act. The internal processes of that organization can use the best of our planning and analysis tools. Still, ultimately, we must allow experimentation and evolutionary processes to guide us. By establishing a rapid learning process that affects individuals, teams and organizations, we can extend the reach of organizations, allowing them to solve highly complex problems.

I appreciate that I am only one human being and my understanding of the world is consequently quite bounded. Still, it is reasonable to hope that some of the concepts discussed here may be of use to you. Others will complement or contradict me as necessary.

The basic concepts that I hope to have contributed an appreciation for are as follows:

• The functional importance of independence, separation and boundaries as counterpoints to the importance of interdependence, communication and integration;
• The trade-offs in scale and complexity, where increasing the set of behaviors possible at one scale (complexity at that scale) requires a reduction in complexity at other scales;
• The need for matching the complexity of the system at each scale to the complexity of the environment (task) at the same scale for the system to be successful;
• The diverse nature of distributed networked systems that are not all the same thing (contrast, for example, the immune system and the nervous system), but can be understood from the same general principles;
• he essential complementarity of competition and cooperation at different levels of organization;
• The constructive nature of both competition and cooperation in forming complex systems;
• The limitations of conventional planning in creating and managing complex systems and the essential importance of planned environments for evolutionary processes;
• The practical utility of fundamental complex systems ideas;

Slightly less apparent but no less important are the recognition and appreciation of:

• the profound paradoxical importance of individual and group differences as a universal property of complex systems;
• the significance of specialization in effective collective behavior, including specialization of individuals and specialization of large subsystems;
• the remarkable emergent behaviors that combine simple capabilities to allow dramatic system capabilities;
• the universal nature of patterns of collective behavior, which serve as elementary building blocks of complex systems just as atoms do;
• the ubiquity of pattern forming processes, differentiation, and particularly local-activation long-range inhibition mechanisms for such patterns.

Finally, along with the recognition of complex problems that we continue to face in this world, we have also pointed out the increasing complexity of society. This increasing complexity implies great capabilities. Indeed, it suggests that we, together, are becoming remarkably effective at solving complex problems in a complex world.”

Making Things Work: Solving Complex Problems in a Complex World, Yaneer Bar-Yam, NECSI Knowledge Press, 2004, 306pp