Social networks and Dunbar's number

In response to my earlier post about Metcalfe's Law, Lior Sion brought up another magic number that is frequently cited when it comes to the number of social relationships: Dunbar's Number. I remembered reading about this in Malcolm Gladwell's Tipping Point and it got me thinking about how it applies to online social networks.

To get started in my investigation, I looked up the Wikipedia article on Dunbar's number. The article describes the number as "a theoretical cognitive limit to the number of people with whom one can maintain stable social relationships". As I continued my search I found a better reference, Dunbar himself explaining how he arrived at this number in this video . According to Dunbar, the number of meaningful relationships a primate can maintain depends on the ratio of the size of neocortex (which exists only in mammals) to the overall brain size. You can compare this ratio and corresponding social unit size in different primates and arrive at a number of roughly 150 in humans. There are several examples of social units without a significant hierarchy that support this hypothesis including Goretex micro-businesses cited in Gladwell's book and Hutterite communes.

So how is this number related to online social communities? Apparently Dunbar is investigating this issue as reported in this article. The full report is due some time later this year. Not surprisingly he notes that even in case of people who have millions of "friends", the communication frequencies indicate that the number of meaningful connections is less than 150. I ran some numbers on Spigit's communities and I came away with the same conclusion. The chart below shows the total number of connections as a function of individual users for three community sizes: small (< 500), medium (< 5000) and large (> 5000). I cut off the long tail in order to clearly show the variation in the fat part (please note that no animals where harmed in this process).

The most conspicuous thing about the charts is the fact that the number of connections as a function of users seems to follow the power law. An interesting property of power law functions is that the natural log of frequencies (y values) varies linearly with log of x values. I did exactly that to the values shown in the first chart and plotted the resulting values in the chart below.

The top few highly connected users fail to live up to the power law expectations (Wikipedia article mentions similar deviations in other phenomena that otherwise follow power law), but the shape of the curve is quite close to being linear towards the end.

Looking at the first chart it is quite clear that the bulk of users have less than 150 connections but what about the highly talkative users that seem to have communicated over a thousand other individuals? The answer lies in the fact that none of these users maintained all these conversations simulteneously. Online communities differ significantly from  the social groups studied earlier in three inter-related respects:

1. Lower barrier to entry - Internet is highly efficient communication mechanism. It is incredibly easy to get virtual access to people that you have no prior knowledge of. 
2. Higher mobility - It is much easier to leave or join a social cluster compared to other social units where physical proximity plays a critical role.
3. Driven by Individual Goals - Online clusters form around a common interest or a goal that is initiated at the individual level not at the communal level. Offline social units form around a set of predefined common goals (e.g. Gore factory unit, HOAs, etc.). Social units (certainly within Spigit communities) in online communities are spawned by people who want to change the world around them.

These three factors contribute to a very dynamic set of social clusters that are formed around topics of interest, gain traction with a small group of users and are eventually replaced by other hot spots with a different membership. In fact looking under the covers, it was clear to me that even highly social users changed their group affiliations over a period of time. Even though they remained interested in communicating with a handful of individuals on a continuous basis, they communicated with different sets of people at different times. The chart shown below illustrates this point. It shows the communication pattern of some of the most connected users in highly successful Spigit communities. Each data point reflects the number of connections in a 30 day period. As you can see from the chart, that number is below 150 with the exception of one data point even though the total number connections over the entire period is above 1000 for some of these users.

So why is this analysis so interesting (other than satisfying one's academic curiosity)? Well, because it helps us evaluate the breadth of collaboration in an online community which is strongly correlated with diversity of participation.  The analysis presented here and in my last post about Metcalfe's law helps us set expectations in this regard. First, the number of interactions per user follows the power law even in the most successful communities. In general the best you can hope to achieve is to lift the long tail  part of the curve upwards. Second, whether it is due to cognitive or bandwidth limitations, the number of active communications in a short time frame that can be maintained by even the most active individuals seems to be below 150. As a result it is not realistic to judge the level of collaboration in terms of the total number of possible connections (n*(n-1)/2), but Dunbar's number seems to be an achievable ceiling for the most active users in your community.

Living up to Metcalf's law on social innovation communities

In my presentation last week at spigit's customer summit, I presented some statistics on spigit's social innovation communities including a comparison between actual connections established over a social network and the maximum possible connections - the basis for Metcalf's law (which is really not a law but we will continue the tradition for the time being). Given the warm reception from the audience at my explanation, I thought I will dig into this a little bit more. This blog post is the result of my investigation.

First let me set some context on Metcalfe's Law. Metcalfe, inventor of the Ethernet protocol, was originally talking about how the "value" of a network of compatibly communicating machines grows quadratically (not linearly). I found this post that shows a reproduction of his original slide. The basis for this assertion is the fact that in a network of "n" nodes, each node can communicate with n-1 nodes hence there are n*(n-1)/2 possible pair-wise connections on such a network. The value of a network is really in its power of establishing communication among its nodes. The number of possible connections is of the order O(n²) as n becomes large which leads to Metcalfe's law.

The mathematics works the same for any network so it is possible to apply the basic thought to other networks including human ones. If each person in an on-line community communicate with all other members, the total number of communication links will be of the order O(n²). Although this may be the ideal scenario, what's the reality like? I performed some analysis on Spigit communities to answer that question. The chart below shows the number of connections in the network plotted against the size of the social network. Three types of connections are plotted:

1. Conversational - Post-response connections between original poster of an idea, blog post, forum thread etc. and a commenter. Note that multiple exchange between two people count as one connection.
2. Conversations and Evaluations - In addition to conversations, this includes voting/rating interactions between two users
3. All connections - These include viewer-poster and linked-in style connections in addition to 1&2

This sanitized version of the graph does not show the numerical data but does depict the trends I observed in my analysis. The chart also shows a plot of linear and nlog(n) functions to give readers an idea about the order of change. Looking at the chart and the numbers behind it, it is clear that the number of conversational connections increases more or less linearly with the community size. The number of connections representing all interactions certainly did not grow at a quadratic rate but the fluctuations were too high to draw any meaningful conclusion. The linear variation in conversational connections is consistent with the long tail pattern of conversations found in all communities. There are a relatively fewer number of individuals who are extremely well connected, but the majority of users are part of the long tail and have a limited number of connections that is independent of the community size.

It is evident from the chart that there is a significant variation in the connectivity utilization between different communities. When I examined the dynamics within more successful communities (in terms of user connections), I found three key factors:

1. Well-defined innovation process -  All successful communities have a very well defined process consisting of idea sourcing, evaluation, and selection. The process is well publicized and executed as promised.
2. Incentives - Most communities have fine-grained incentives for encouraging broader participation in addition to allocating funding for implementing selected ideas. The incentives range from tangible items like gift cards to recognition by leadership to purely monetary rewards corresponding to the play money earned in Spigit's virtual economy.
3. Emphasis on "social" - Some of the well connected communities emphasize social interactions by promoting blogging, open exchange of thoughts on discussion forums, and (now) traditional collaboration over wikis. Innovation is embedded within this larger context.

It is interesting to note that not every successful community incorporated all three factors. Also in general increased connectivity did not necessarily lead to a better result in terms of number of quality ideas that get selected for implementation. That is strongly correlated only to the first two factors. Conversely in some communities,  organizational culture inhibited people from freely collaborating on proposed ideas, but the outcome has been significantly better compared to traditional processes due to support from the executive team and strong innovation process management. This is an important point to consider when setting up enterprise communities. Whereas communication within consumer-oriented social networks is very often an end in itself, for enterprise communities communication is a means to an end. For spigit communities, the goal is the selection and implementation of ideas that provide business value and achieving that requires channeling community interactions to that end.

It's also worth mentioning a few patterns that I thought I would see but didn't. I didn't find any correlation between different business sectors and network connectivity nor did I find more interactions within communities that had higher percentage of knowledge workers. There was C-level involvement in a couple of cases but in many communities the innovation effort was lead by an internal champion who was passionate about making the innovation effort successful. Finally the size of the community didn't seem to have much effect on the networking pattern even though it did have an effect on the number of participants that posted on the innovation site.