Padgett, Emergence of Organizations and Markets, Part I

I'm an idea junkie. I have fairly diverse interests and scan lots of things. But, for the most part, the things I come across strike me as either generally similar to this or that idea I've come across before or interesting and novel but focused on a matter that isn't terribly consequential.

It is very rare that I come across something that strikes me as fundamental -- in the sense that it addresses a deep problem in a novel way that makes intuitive sense to me. I count three instances in the past decade:
1) Holling's conceptualization of the operation of systems in terms of panarchy.
2) Abbott's description of the process of knowledge change.
3) The ideas discussed in the rest of this post.

How do new organizational forms originate? As a visit to the Oil Drum, or virtually any other major environmental website shows, you see lots of calls for dramatic social transformation. And you can also find good studies documenting historical change in organizational form, for example Chandler's analysis of capitalist managerial organization. But these works, in a Darwinian sense, focus on the selection process -- why a novel form gets perpetuated -- rather than the innovation process -- how the novel form originated. While there is some discussion of where novel ideas come from, the focus tends to be on technological innovation. In many ways it makes sense to speak of bureaucracy as a technology, but it is profoundly problematic to think that the process of social innovation is just a mirror of the process of technological innovation. So, to summarize, a general recognition of the need for social innovation -- that is the development of new forms of social organization -- coupled with virtually no understanding of how novel forms of social organization are generated.

Into this gaping hole walks John Padgett and his collaborators with their forthcoming Princeton University Press title The Emergence of Organizations and Markets. In a work of stunning scope, they lay out a general theory of organizational innovation and then proceed to provide a diverse set of illustrative examples covering the sweep of modern history -- from early capitalism (the emergence of merchant banks, the partnership system, global markets and the joint stock system), thru studies of communist economic transition (China, USSR, Hungary), to the emergence of new linkages between science and capitalism (e.g., the emergence of Silicon Valley and other high tech research network clusters). While the book focuses on economic organization, there is no obvious reason the theory wouldn't apply to other types of organization.

In essence, Padgett translates the biochemical theory of autocatalysis, which Stuart Kauffman has argued provides a model for abiogenesis (the emergence of biological life out of inorganic matter), into a form useful for understanding the emergence of organizations. There are three key parts. First, there is the emergence of novelty. In the abiogenesis example, this would be the emergence of life out of inorganic chemicals. Second, there is the requirement of persistence through time. If life emerged, but failed to persist over many generations, then its development would be transient and not terribly important. Third, the new development must be consequential. The emergence of a single-celled living organism that persisted for millions of years but remained the only form of living organism, as significant as that example of life may be, is not as consequential as the development of a wide variety of diverse living organisms (cats, dogs, whales, people, birds, insects, etc.). I'm not talking about the value of the different organisms themselves, but rather the importance of a process which spills over into other areas and generates greater and greater variety.

In the process, Padgett comes up with a new way of thinking about individual humans: not as static, bounded objects but, rather, as dynamic flows of chemicals, energy and information embedded in networks. The implications for scholars interested in understanding conjoined socio-ecological systems, which have been plagued by the attempt to integrate ecology's focus on flows (energy flows and biogeochemical cycles) with approaches to human systems that emphasize the primacy of individuals and their actions, are hard to overestimate.

Here is how they organize the theoretical discussion:

First, we describe the problem of organizational novelty in the context of multiple social networks. Next, we explain our core dynamic motor of autocatalysis, at the levels of both chemical and economic production. Then, we extend the biochemical concept of autocatalysis to encompass the social production of persons. Fourth, we describe six network mechanisms of organizational genesis that we have discovered in our case studies. Fifth, we identify seven mechanisms of multiple-network catalysis that turned these organizational innovations into systemic inventions. Finally, we discuss the important outstanding issue of structural vulnerability to network tipping. This question of poisedness, for us, is the next research horizon.

There is way too much here for one post. So, there will be several spread out over time. If you can't wait, the key theoretical chapter is here. Padgett's homepage has links to other material, both published and unpublished, including several other chapters from the forthcoming book.  For an extended overview, including description of the multi-network perspective,  read on ....

Key Concept: Emergence or Those flocking boids!

One of the great philosophical arguments of all time involves conceptualizing the relationship between different levels of phenomena: does one favor organicism or reductionism? Is the whole greater than the sum of its parts? Or can it be reduced to an understanding of those parts?

Sociology, following Durkheim and his 'social facts can only be explained through reference to other social facts' dictum, traditionally favored organicism. One of the more interesting places to explore these ideas is by looking at emergence. How, for example, do we understand the flocking of birds or the existence of segregated neighborhoods in cities?


Craig Reynolds has a mesmerizing page where you can watch an individual based model of boids that mimic the flocking behaviour of birds. In philosophical terms, Reynolds is trying to explain the emergent phenomena of flocking behaviour through a set of individual-level instructions to his computerized agents (the boids). The page does an excellent job of explaining both the assumptions that are programed into the agents and providing links to a wide variety of relevant literature.

This is similar to the famous explanation for segregation provided by Nobel prize winning economist Thomas Schelling in his book Micromotives and Macrobehavior. By positing the existence of individuals (agents in the jargon) with slight-but-not-malicious preference to have neighbors of the same race, Schelling developed an agent based model showing that, over time, the result of people acting to meet these preference leads not to a mixed neighborhood but, rather, to completely segregated populations. There are a number of different simulations of this on the web, for example, here, here and here. A simpler and less interesting version is here.

But, as Russ Abbott has noted, while boids do a good job of modeling flocking behavior, they ignore energy issues. Thus, as currently conceived, most agent based representations of ecosystem phenomena are seriously deficient.