Padgett, Part II: Emergence of Partnership

A previous post described the basic outline of one of the most important ideas I've come across in the past decade: Padgett's use of multi-network perspective to explain organizational invention. This post provides a concrete illustration of that perspective in action through the emergence of a system of economic partnership in Renaissance Florence. Future posts, exploring the theory in more abstract terms, will refer back to this material as a way of making the ideas more understandable. For more details .... read on.

Padgett, Emergence of Organizations and Markets, Part I

I'm an idea junkie. I have fairly diverse interests and scan lots of things. But, for the most part, the things I come across strike me as either generally similar to this or that idea I've come across before or interesting and novel but focused on a matter that isn't terribly consequential.

It is very rare that I come across something that strikes me as fundamental -- in the sense that it addresses a deep problem in a novel way that makes intuitive sense to me. I count three instances in the past decade:
1) Holling's conceptualization of the operation of systems in terms of panarchy.
2) Abbott's description of the process of knowledge change.
3) The ideas discussed in the rest of this post.

How do new organizational forms originate? As a visit to the Oil Drum, or virtually any other major environmental website shows, you see lots of calls for dramatic social transformation. And you can also find good studies documenting historical change in organizational form, for example Chandler's analysis of capitalist managerial organization. But these works, in a Darwinian sense, focus on the selection process -- why a novel form gets perpetuated -- rather than the innovation process -- how the novel form originated. While there is some discussion of where novel ideas come from, the focus tends to be on technological innovation. In many ways it makes sense to speak of bureaucracy as a technology, but it is profoundly problematic to think that the process of social innovation is just a mirror of the process of technological innovation. So, to summarize, a general recognition of the need for social innovation -- that is the development of new forms of social organization -- coupled with virtually no understanding of how novel forms of social organization are generated.

Into this gaping hole walks John Padgett and his collaborators with their forthcoming Princeton University Press title The Emergence of Organizations and Markets. In a work of stunning scope, they lay out a general theory of organizational innovation and then proceed to provide a diverse set of illustrative examples covering the sweep of modern history -- from early capitalism (the emergence of merchant banks, the partnership system, global markets and the joint stock system), thru studies of communist economic transition (China, USSR, Hungary), to the emergence of new linkages between science and capitalism (e.g., the emergence of Silicon Valley and other high tech research network clusters). While the book focuses on economic organization, there is no obvious reason the theory wouldn't apply to other types of organization.

In essence, Padgett translates the biochemical theory of autocatalysis, which Stuart Kauffman has argued provides a model for abiogenesis (the emergence of biological life out of inorganic matter), into a form useful for understanding the emergence of organizations. There are three key parts. First, there is the emergence of novelty. In the abiogenesis example, this would be the emergence of life out of inorganic chemicals. Second, there is the requirement of persistence through time. If life emerged, but failed to persist over many generations, then its development would be transient and not terribly important. Third, the new development must be consequential. The emergence of a single-celled living organism that persisted for millions of years but remained the only form of living organism, as significant as that example of life may be, is not as consequential as the development of a wide variety of diverse living organisms (cats, dogs, whales, people, birds, insects, etc.). I'm not talking about the value of the different organisms themselves, but rather the importance of a process which spills over into other areas and generates greater and greater variety.

In the process, Padgett comes up with a new way of thinking about individual humans: not as static, bounded objects but, rather, as dynamic flows of chemicals, energy and information embedded in networks. The implications for scholars interested in understanding conjoined socio-ecological systems, which have been plagued by the attempt to integrate ecology's focus on flows (energy flows and biogeochemical cycles) with approaches to human systems that emphasize the primacy of individuals and their actions, are hard to overestimate.

Here is how they organize the theoretical discussion:

First, we describe the problem of organizational novelty in the context of multiple social networks. Next, we explain our core dynamic motor of autocatalysis, at the levels of both chemical and economic production. Then, we extend the biochemical concept of autocatalysis to encompass the social production of persons. Fourth, we describe six network mechanisms of organizational genesis that we have discovered in our case studies. Fifth, we identify seven mechanisms of multiple-network catalysis that turned these organizational innovations into systemic inventions. Finally, we discuss the important outstanding issue of structural vulnerability to network tipping. This question of poisedness, for us, is the next research horizon.

There is way too much here for one post. So, there will be several spread out over time. If you can't wait, the key theoretical chapter is here. Padgett's homepage has links to other material, both published and unpublished, including several other chapters from the forthcoming book.  For an extended overview, including description of the multi-network perspective,  read on ....

Emergence: Spontaneous Order Theory

Most of the material I've read about emergence traced its conceptual roots to the natural sciences -- even when dealing with social creations like cities or the world wide web. But there is a significant tradition focusing on 'spontaneous order' traceable to Adam Smith (known for the concept of the Invisible Hand) and the Scottish Enlightenment. So, here is some links to that material

1) The online journal Studies in Emergent Order

2) A couple of papers in the journal that look particularly interesting:

“Conflicts and Contradictions in Invisible Hand Phenomena”

This paper makes three interconnected arguments.

• Building off my other writings, in a world where there are many non-teleological complex adaptive systems exist, no automatic harmony exists between their different coordinating processes. This paper will focus on four of these systems: the ecological system at the landscape level, and three cultural systems, the market, science, and democracy.
• Organizations originating within one such system but operating within more than one, will ultimately be dependent on one set of feedback signals over the others. When conflict between sets of signals arises, such organizations will disrupt, undermine, or destroy the other ordering processes.
• Therefore a system of Hayekian spontaneous orders such as the market, democracy, and science, is not and cannot be sustainable based solely on their own internal characteristics because conflict between them is an intrinsic feature of social life. The same hold for any of these systems and an ecosystem. They need to be viewed within a larger context.

“From Hayek’s Spontaneous Orders to Luhmann’s Autopoietic Systems”

In this paper I contrast Hayek’s and Luhmann’s treatment of law as a complex social system. Through a detailed examination of Hayek’s account of law, I criticize the explanatory power of his central distinction between spontaneous order and organization. Furthermore, I conclude that its application to law leads to different results from the ones derived by Hayek. The central failure of Hayek’s failure, however, lies in his identification of complex systems with systems of liberal content maximizing individual freedom. Indeed, in this way, he can only account for systems-individuals and not systems-systems interactions. I introduce Luhmann’s theory of autopoietic systems, which I submit, can solve all the mentioned problems and seems a much more promising conceptual architecture to grasp social systems in the context of a complex society.

3) An annotated bibliography of works on self organizing systems

4) An article tracing the intellectual history of the tradition: From Smith to Menger to Hayek: Liberalism in the Spontaneous-Order Tradition

On the Royal Wedding, Emergence and Political Discourse in the US and Britain

I had no plan to watch the royal wedding this morning, but it happened to coincide with my morning coffee. And, as I watched, I was struck by a peculiar philosophical difference between American and British political discourse and its relation to the concept of emergence.

While the wedding had lots of pomp and ceremony, I was surprised at the clear, explicit and didactic nature of the statements made by the Cardinal. He literally provided numbered reasons for the benefits of marriage ... one, to have children; two ...; etc. It was not just a ceremony for those involved, it was also a message to the British about the intended role of the Royals. And, looking at the larger structure of the discourse, here is what was articulated:

1) The wedding ceremony involved a 'fusion' of William, Kate and God. Significantly, the point was made explicitly and repeatedly that the result of this process -- making God an operative element in their lives -- should not be seen as forcing them to 'conform' to a particular moral or ethical code. Rather, fusing themselves with God was meant to be transformative -- to provide them with access to the Holy in a way that allows them to navigate novel situations with a moral compass informed by God about what is good and what is not.

2) Later in the ceremony, there was some explicit discussion of the future and, in particular, the obligations of the Royals in facilitating it. This was, as one would expect in a nation where formal political decisions rest with the elected officials in Parliament, a bit more subtly phrased -- but the intended meaning was still clear. The Royals, using their moral compass informed by God, have an important role to play in defining the direction of British society and, interestingly, its relation to nature.

I don't know squat about Anglican doctrine, previous royal weddings or any particular role that Charles (and his well known green bent) might have had in crafting the reference to preserving nature. But the general philosophical tone of the ceremony was clear: rather than leading through adherence to a rigid code, the Royals are expected to channel God and provided enlightened and emergent leadership that addresses novel problems in novel ways.

This view is, interestingly, in stark contrast with American political discourse where the dominant theme has to do with the infinite wisdom of the founding fathers and the merit in finding solutions that conform to their ideas. This orientation is most explicitly present in the legal philosophy of 'strict constructionism' but the invocation of the founding fathers is a standard trope in Congress as well.

So this is where the English speaking world currently finds itself. On one side of the pond, moral leadership is invested in an in-bred group of Royals who are expected to channel God in order to help Britain find its way. But they have no formal political power. On the other side of the pond, God has been replaced by a group of long dead men -- the founding fathers -- as the source of enlightened governance. Moreover, rather than adaptively addressing new problems, solutions are found though conformity to the ideas of the founders. Personally, I don't find either of those options -- adaptive leadership based on the Divine right of Kings and the spiritual connection of the Royals to God or rigid doctrinal adherence to a civil authority constrained to the ideas of a group long dead men -- particularly attractive. Can't we some how find a way to get adaptive governance integrated into a civil authority structure?

Documentary: Secret Life of Chaos

Interesting documentary on the chaotic systems, particularly the process of self-organization and emergence. Some really nice visuals illustrating the processes as well.

Crowd Sourcing the Climate: An assessment of Climate Co-lab

A recent post by Andrew Revkin brought the existence of MIT's Climate CoLab to my attention. Here is a description of the project by its Director, Thomas Malone at MIT's Center for Collective Intelligence.



I approached the site with high expectations. Claiming inspiration from Wikipedia and Linux, obvious examples where effective use of distributed intelligence has facilitated the creation of an emergent product, the site aims to split the climate problem into manageable parts and foster productive online discourse aimed at distilling solutions. Unfortunately, it seems they have a long way to go.

First off, there aren't a lot of participants and those who are there are organized competitively rather than cooperatively. The base structure involves teams that propose plans which are then adjudicated in a two phase process -- by experts for feasibility (because earlier plans chosen by the community turned out not to be feasible) a then by a vote among the community. In other words, it is a contest. As a result there are a number of proposed plans (22 when I looked, only 8 of which had attracted any support from the community) generated by team members (i.e., you have to be a member of the team to work on the proposal). And the teams are small (3 members or fewer as near as I could tell). In other words, the site is structured to generate competing (and hopefully, insightful, ideas) rather than work collaboratively on the problem. Or, stated another way, the premise is that some genius off in cyberspace can come up with an idea the rest of us haven't thought of. But that, of course, is precisely the problem with climate change. It is massively complex and, for precisely that reason, seems to lend itself to a LARGE SCALE collaborative effort like Wikipedia or Linux. Moreover, most of the teams seem to be there to push specific ideas that have been proposed elsewhere rather than to develop new ideas of their own.

Second, the "discussion" consists of a) voting on a number of propositions that the site authors have put up (e.g., Is the earth's climate changing because of human activity? Yes or No?)and b) making comments (almost no one has). Again, this doesn't seem like a serious effort to move the debate forward.

But most troubling, from my perspective, is that these supposed experts in Collective Intelligence seem to be pretty clueless about the role of social structure in collaboration. On the one hand, they have written an interesting article (The Collective Intelligence Genome) in which they claim "Collective intelligence has already been proven to work" (I agree) and "CI systems can be designed and managed to fit specific needs (through the use of) CI building blocks, or “genes,” (that) can be recombined to create the right kind of system" (an interesting idea that recognizes the connection between structure and success).

But the "gene" they have decided to use to solve the climate problem is the one best exemplified in the success of Threadless, a model they describe as follows: "anyone who wants to can design a T-shirt, submit that design to a weekly contest and then rate their favorite designs. From the entries receiving the highest ratings, the company selects winning designs, puts them into production and gives prizes and royalties to the winning designers. In this way, the company harnesses the collective intelligence of a community of over 500,000 people to design and select T-shirts."

This is a great model (like You-Tube) for getting other people to do free labor for you in the development of a product. But anyone who thinks designing a t-shirt is a problem with anything remotely approximating the complexity of solving the climate change problem they need their head examined. The folks at ClimateCoLab would have been much better off re-reading Andrew Poe's insightful article (The Hive) about the evolution of the social rules and structure responsible for Wikipedia's success. Any serious attempt to deal with the climate problem through the collective intelligence process lies down that path and not down the one being touted by ClimateCoLab.

Is there such a thing as collective intelligence?

There is a long running debate in psychology over whether or not there is such a thing as general (or g-factor) intelligence. Charles Spearman, in the 1920's, proposed the existence of such a factor as a way of explaining why schoolchildren's grades in seemingly unrelated subjects were correlated.

A recently published article in Science (DOI: 10.1126/science.1193147) extends this concept to groups of people, arguing that groups have a “collective intelligence” that predicts their performance on a range of collaborative tasks. The study, of 699 people working in small groups, found that groups which performed well on one type of collaborative task tended to perform well on others while groups that performed poorly on one type of task tended to perform poorly on others. They found that neither the intelligence of any particular group member nor the average individual intelligence of the group members were particularly good predictors of group behaviour. Instead, the study found the degree to which group members were attuned to social cues (e.g., recognizing whether other group members were annoyed or worried) and their willingness to take turns speaking were more important, as was the proportion of women in the group.

Personally, I have huge problems with the specifics of this type of study -- where grandiose claims are made based on artificial tests that bear little resemblance to real world situations. More fundamentally, if such a group trait exists, it is likely to be an emergent phenomenon rather than one that can be explained in reductionist terms (e.g., through reference to the individual psychological traits and simple interactional processes identified in the article). That said, they deserve credit for revisiting the study of small groups (which sociology largely abandoned after the 1950's) and for their emphasis on understanding collaborative behaviour. All in all an interesting supplement to Elinor Ostron's work on processes of collaborative governance.

Clouds not Clocks

Jonah Lehrer's recent article in Wired surveys developments in a variety of fields, from neuropsychology to genetics to particle physics, to argue against reductionism.

Karl Popper, the great philosopher of science, once divided the world into two categories: clocks and clouds. Clocks are neat, orderly systems that can be solved through reduction; clouds are an epistemic mess, “highly irregular, disorderly, and more or less unpredictable.” The mistake of modern science is to pretend that everything is a clock, which is why we get seduced again and again by the false promises of brain scanners and gene sequencers. We want to believe we will understand nature if we find the exact right tool to cut its joints. But that approach is doomed to failure. We live in a universe not of clocks but of clouds.

Forests and Carbon in Canada

Deforestation, estimated to be responsible for up to 20 percent of global human generated emissions contributing to climate change, has long been a major focus of climate change policy. Most of the emphasis, however, has been on tropical deforestation. The reasons for this are fairly straightforward: 1) tropical forests are much more effective carbon sinks than are temperate forests (they grow faster and, hence, remove carbon at a more rapid rate) and 2 the volume of forest being cut is larger in the tropics than in temperate regions.

However, recent work by Werner Kurz and others in the Canadian Forestry Service has shifted attention to the forests of British Columbia and surrounding areas which are being devastated by an outbreak of the pine beetle.




According to the Montana Department of Environment, the mountain pine bark beetle (a species native to the region) has benefited from a variety of climate related changes:

1) Prolonged periods of cold weather in the -20 to -40 F range kill pine bark beetle larvae embedded in the thin bark of Lodgepole trees. However, winter weather over recent decades has afforded very few of these extended cold snaps. ... 2)Recent late summer drought conditions have stressed Lodgepole forests. Weakened individual trees have long been known to issue a biological invitation to beetle attack. During its adult mating phase in late summer, adult scout beetles identify weakened trees and issue an aggregating pheromone to other beetles to key-in and attack the tree. The apparent combination of large numbers of drought-weakened trees and the growing population of beetles perpetuates the infestations.

As a result BC's forests, which have traditionally been viewed as a carbon sink, have become a major source of CO2 emissions. As shown in the diagram below, Canadian Forestry estimates now identify the pine beetle infestation as the single largest cause of CO2 emissions in Canada, exceeding by a significant amount the substantially more criticized emissions from the oil sands.



As a result, BC is now embroiled in a policy debate about how to proceed: harvest and burn the wood rapidly (which implies a big hit in short-medium term C02 emissions from the burning in order to get the forest back acting as a carbon sink through active reforestation) versus letting it decay slowly (and, hence, having a lower rate of emissions over a longer period coupled with a longer wait for the forest to return to its role as a carbon sink). The details of these, and other related debates, are nicely summarized in a recent article in the January 9, Globe and Mail, From Green Hero to Carbon Villain.

At a more fundamental level, these developments underscore the uncertainties inherent in attempting to devise a carbon accounting system for a planet that is in the process of profound transformation due to climate change. A few short years of warm winters and dry summers and a whole region goes from being a major carbon sink to a major source of emissions.

Approaches to Complex Systems

In this talk George Francis provides an overview of different approaches to the study of complex adaptive systems. The talk is interesting in that he draws as much on social science theorizing as that developed in the natural sciences. World System Theory, in particular, is discussed at length.

He is also the author of 'Models for Sustainability Emerge in an Open Systems Context,' an article covering much the same ground.

Climate Change as Evidence of A System

Where can we find evidence of a social system? Do we have to concoct some kind of academic hocus-pocus to linguistically conjure up the "idea" of a social system?

I don't think so. Evidence of a system, and by that I mean a human social system, is all around us: it's climate change.

Let's look at what we already know from the science of climate change:

• Individuals don't create climate change

• 6.5 Billion individuals together don't create climate change

However:

6.5 Billion individuals PLUS a capitalist system, a technological system, create climate change.

In fact, it isn't even all 6.5 billion of us. At least half the world's population is so poor, so "underconsuming," that their collective contribution to global carbon emissions is almost negligible.

It really amounts to the richest one sixth of the world's population, the 1 billion or so who consume the most, who contribute the vast majority of the world's excessive carbon emissions. And it's not just that 1 billion alone. It's 1 billion PLUS all their automobiles, highways, coal-fired power plants, suburban subdivisions, millions of acres of shopping malls, agribusiness, distant factories and 60" television sets, all consuming, burning and emitting energy, that together create climate change. The technological system of resource extraction, production, distribution and consumption, as an economic system, makes a larger contribution to the production of climate change as the billion or so "individuals" who participate in that system.

To this we must add "nature's own" climate feedback cycles that, triggered by this excessive human C02 production, continuously creates more warming than just the human social system alone. What begins as a human system problem then becomes a human/environment system problem. And that is the ultimate designation of a system.

And to prove this point, please listen to physicist Anita Burke, who worked for Shell Oil International, advising their top corporate leaders on the causes and impacts of climate change. She spoke at the "Gaining Ground" conference in Vancouver, Oct. 2009, along with Bill Rees. Anita gives us the numbers: individual and household consumption of energy, water and other resources only counts for 10-25% of all resource consumption. The other 75-90% is used by corporations.

Key Concept: Emergence or Those flocking boids!

One of the great philosophical arguments of all time involves conceptualizing the relationship between different levels of phenomena: does one favor organicism or reductionism? Is the whole greater than the sum of its parts? Or can it be reduced to an understanding of those parts?

Sociology, following Durkheim and his 'social facts can only be explained through reference to other social facts' dictum, traditionally favored organicism. One of the more interesting places to explore these ideas is by looking at emergence. How, for example, do we understand the flocking of birds or the existence of segregated neighborhoods in cities?


Craig Reynolds has a mesmerizing page where you can watch an individual based model of boids that mimic the flocking behaviour of birds. In philosophical terms, Reynolds is trying to explain the emergent phenomena of flocking behaviour through a set of individual-level instructions to his computerized agents (the boids). The page does an excellent job of explaining both the assumptions that are programed into the agents and providing links to a wide variety of relevant literature.

This is similar to the famous explanation for segregation provided by Nobel prize winning economist Thomas Schelling in his book Micromotives and Macrobehavior. By positing the existence of individuals (agents in the jargon) with slight-but-not-malicious preference to have neighbors of the same race, Schelling developed an agent based model showing that, over time, the result of people acting to meet these preference leads not to a mixed neighborhood but, rather, to completely segregated populations. There are a number of different simulations of this on the web, for example, here, here and here. A simpler and less interesting version is here.

But, as Russ Abbott has noted, while boids do a good job of modeling flocking behavior, they ignore energy issues. Thus, as currently conceived, most agent based representations of ecosystem phenomena are seriously deficient.