My three principles of new product development

Spigit recently released a Facebook application that exposes Spigit's social innovation management functionality on the platform. It is always thrilling when something you help create sees the light of the day. In this case it was particularly satisfying because we followed my three principles of new product development with a great degree of success (thanks to FacebookEngage product manager Nina Chai and Pratim Mukharjee, the lead developer on the project).

Simple but not Simpler
I have lead the evolution of the Spigit platform from an abstract concept to the leading social innovation platform used by over 100 organizations all over the world. Initially we had to take a leap of faith in creating the first version of the product. Combining idea management with social networking was a new concept for a lot of our prospective users. We certainly made a lot of untested assumptions to create that first version. We did strive, however, to minimize the number of assumptions and create something that was simple (with a minimal feature set) but not simpler, i.e. it provided unprecedented benefit to the end users. We have started Spigit's Facebook application development much the same way. The initial version exposes minimal functionality. It's a proposal to our customer base which they use, review, and evaluate.The rest of the course will be charted by combining customer feedback with our own intuition and understanding of the marketplace.

Native Experience
There are some products that are so revolutionary that they change existing interaction paradigms, processes, or even the way of thinking. Most others need to seamlessly fit into existing  lifestyles and usage conventions. They must blend with established usage patterns and feel completely native to the user. We had to be very careful about this blend for the facebook application. A large portion of Spigit's user base is in the enterprise space. Facebook users are significantly different in their expectations, attention spans, and motivation to participate. Spigit's Facebook application deliberately sports a non-enterprise, lightweight look. The primary motivation of Facebook users is keep in touch with their social circle. The Spigit apps plays into that by making the user aware of ideas posted by your friends and how many of them made into the top ranking list.The ideas are displayed very much in a wall-style format. When a user posts an idea, posting that on the wall or sending message can be done with minimal effort. In general, we have made every effort to introduce Spigit functionality in a way that is consistent with ethos of Facebook community.

The Wow Factor
A new product should have at least one or two features that make a user go "Wow that's useful!" when s/he uses the product. When I started using iPhone, I had that reaction when I realized that unlike my previous mobile windows-based phone, the screen goes dark when I put it to my ear but it comes alive when I hold it in front of me. With Spigit's facebook launch we did not have a problem introducing the Wow factor. We pulled in some of Spigit's key features deemed relevant to Facebook. We included idea theme clusters that allows administrators to get a sense of emerging trends and topics. Reputation scores identify key contributors in the community. Spigit's social analytics is available to the administrators via Spigit's regular user interface. Subsequent versions of the app will track user feedback and will build upon the initial feature set to make it even better.

Thoughts on spigit, idea management, and future of social innovation

Javed Mohammad, a community manager at Spigit, interviewed me recently. This blog post simply the transcript of that interview. The conversation has also been posted on the Spigit blog.

JM Before companies like Spigit came along how was idea management handled?


PD Before spigit, idea management was mainly done in a very top-down manner. A very small set of technology and business leaders were responsible for innovation. The flow of ideas was mainly uni-directional, going from the leadership to the rest of the organization.

JM And what was the implication of that?

PD The process was very rigid and lacked transparency. We have now seen several instances where so called experts make decisions that turn out to be worse than those made based on input from a diverse set of people. The universe of ideas was also vastly smaller since the process relied on a very small group of people.

JM Somewhere along the line you got the inspiration for Spigit. What was it?

PD At the time we founded Spigit, social networking was already popular in the consumer space. We had seen isolated instances of blogs and wikis that went beyond just the social chatter and attempted to provide business value. For example, we were noticing a lot of bloggers proposing interesting ideas with business potential. The original concept was to create a product that would facilitate turning these raw ideas into real products and services.

JM How did you combine what would seem to be two incompatible applications one consumer and the other business into a product like Spigit?

PD That's really our key distinction. When we started Spigit, you either had consumer-facing applications that supported online social networks or you had traditional enterprise applications that supported very opaque and rigid processes like Stage Gate. We wanted to retain and leverage the social, fun aspect of consumer oriented applications, but at the same time channel that social energy in a very productive way. Secondly, we needed an engine for analyzing social exchanges and create actionable intelligence for our enterprise sponsors.

JM How does one provide “fun” in an Enterprise application?

PD Introducing "fun" in enterprise applications is getting very popular these days. In face now there is a name for it: "Gamification". Gamification is the application of game design principles to non-game applications to influence user behavior in accomplishing real life tasks. It is about creating a fun, engaging experience for users that encourages them to perform tasks that they might otherwise find tedious or common place. Frequent flyer programs is a classic example of how gamification helps build loyalty. In spigit, we realize it via leaderboards, user badges, reputation, idea markets, etc.

JM We hear a lot about “the wisdom of crowds.” Who or which type of organizations have been able to leverage this wisdom?

PD Any organization where there is a common goal and collaboration of its community. They have to be open, transparent and able to execute. As long as they are willing to implement ideas that come through the idea funnel, and the community sees that as a reinforcing behavior, those are the organizations that thrive.

JM Can you give any examples that are public?

PD AT&T has a company-wide innovation program built on top of the spigit platform. That's an example of an internal idea management program that is very democratic. Cisco's I-Prize that was looking for the next billion dollar idea is an example of external facing application of Spigit's technology. I-Prize is also an excellent example of the power of idea markets, one of the key gamification features in spigit. The top three ideas predicted by the markets were exactly the same as the ones selected by Cisco.
JM Gartner in their Hype cycle for emerging technologies shows Idea Management as having crossed the trough of disillusionment. When do you think it will reach the plateau of productivity which I guess is Nirvana?

PD Idea management, the way Spigit pioneered it, began about 4 years ago. I have seen a measurable change in the attitude of enterprise decision makers towards this technology. Questions have changed from "should we" to "which tool". I think it would still take one or two more years to reach maturation and it may not reach the plateau in isolation. Idea Management will most likely be combined with other business processes like Talent and Resource Management, CRM, etc.

JM What excites you about the field of social innovation and what’s your prediction for the future of ideation?

PD The current solutions have done a good job of decentralizing idea collection process. We could definitely improve collaboration and idea evolution by providing intelligent matchmaking between potential collaborators. In many cases, ideation framing is still done by a central group. There is still a single sponsor and they are responsible for doing selection and implementation. That part needs to be decentralized. Fundamentally, it is all about collaborative problem solving and the future should enable a decentralized, peer-to-peer communication among seekers and solvers.

JM Thank you.

Essential features of a scalable social innovation platform

The overall outlook and expectations towards social business software have changed dramatically over the last few years. Most enterprise decision makers have stopped questioning the value of social technology and are now more concerned with finding the right enterprise social platform that can support large communities within and beyond organizational boundaries. The trend is especially stronger in open innovation (or social innovation as I like to call it) communities. As a result, a number of existing players in innovation management have re-focused their product strategy to incorporate social elements. We are also seeing a large number of new idea management products entering the market at a rapid pace.

Although it seems that idea and innovation management tools have become a commodity, the reality is that most tools are limited to running campaign style idea collection leaving idea evaluation, filtering, and ranking in the hands of a small group of sponsors. This approach may be sufficient for some of the smaller ideation campaigns but does not scale very well when the community size and the number of ideation efforts go up. In order to be scalable, a social innovation platform must enable sponsors and community managers to:

• Clearly frame ideation objectives and engage the right audience for optimal results
• Manage online behavior of community members such that meaningful contributions are promoted and spamming/gaming attempts are minimized
• Ensure diversity of participation and connect the right set of collaborators belonging to the long tail
• Create an engaging experience that keeps users coming back
• Leverage the crowd to filter and rank ideas to minimize bias and management overhead
• Choose the right balance between crowd sourcing and top-down control

In this blog post I outline a core of set of features that must be present in a social innovation platform in order to effectively meet the needs listed above. Note that I focus only on advanced features that go beyond basic functionality such as idea posting, commenting, and voting.

Multi-tenant architecture and templates
Innovation efforts in large organizations involve multiple regions, lines of businesses, and ideation goals. We certainly recommend creating a global ideation site that allows the entire community to collaborate across organizational silos, however it is also necessary to create destinations visited by different population segments. The social innovation platform must allow sponsors to easily set up new communities and invite relevant sections of the user population for participation. Moreover it must allow innovation leaders to set up community templates with terminology, look and feel, evaluation and ranking mechanisms representing different ideation objectives and policies. Community templates significantly reduce the barrier to entry for new ideation sponsors; accelerates the learning curve and thereby minimizes effort required by the core innovation team to bring the new entrants up to speed.

User reputation
Innovation communities face two opposite types of problems: users posting too many ideas or an inactive community with too few ideas. The root cause in most cases is improper ideation framing. Some large communities, however, suffer from the former problem in spite of having a dedicated innovation team that clearly defines the goals. When the noise drowns out the substance, it makes it more difficult to make productive contributions and drives away serious users. Computing user reputation is an effective strategy to cope with this problem.

It is important to note that reputation is a reflection of quality of contribution not quantity. Some idea management tools use the term reputation to indicate activity level. Reputation rank should be determined not by how many times a user posts on the community, but how he community reacts to the user's online behavior and participation.

The reputation ranking of users has three implications for community management. First, displaying reputation on the leaderboard is a way of giving social recognition to your top contributors. It also creates a sense of competition, a great motivator for almost every kind of user. Second, the reputation score can be factored in idea ranking greatly improving relevance. Third, if reputation is tied to currency or points earned by users, it encourages users to be active in a meaningful way.

Reputation can also be used as a stick. The innovation platform must detect spammers and users who attempt to game the system. Reputation score for such users should be dropped to zero or a very low number thereby limiting their ability to get points and influencing idea ranking.

Game mechanics
The term game mechanics or "Gamification" is becoming quite popular in the context of social media discussions. This obviously does not translate to introducing Farmville to your community. That will probably result in having the "too few ideas" problem that I mentioned earlier. Gamification should really be about creating a fun, competitive environment for your innovation community that makes the site sticky in a positive manner. It should be designed in such a way that winning the game requires participants to align their behaviors with innovation objectives and strive to improve overall community health.

Gamification goes much beyond getting Foursquare badges (although that could definitely be part of it). It should include leaderboards that displays users by reputation, virtual wealth, and top innovators in addition to different types of idea rankings. Users should be able to acquire special powers that allows them to moderate content, view restricted areas or access functionality not available to other users. Mechanisms like idea markets and prediction markets are another example of constructive gamification. These mechanisms are not only a fun way of engaging the crowd, but they also enable sponsors to aggregate opinions of many and derive results that assist in decision making and making predictions.

Automated idea graduation
I recently heard a great quote about the value of crowdsourcing:

It's not difficult to find a needle in the haystack if you enlist the haystack to find it.

The point here is that it's a lot easier to find great ideas and solutions when you open up ideation challenges and problems to the entire community. I believe in going even further and enlisting the crowd in filtering and ranking of ideas in addition to initial suggestions. When you leave idea evaluation and ranking entirely in the hands of the sponsors, it is problematic in a couple ways. First, you really expose yourself to the biases shared by a small group of people. The HiPPO problem (highly paid persons opinion) is a corollary of this issue. Second, the innovation committee becomes a bottleneck in the innovation process as the number of ideas increase.

The innovation platform must support a way of filtering ideas in multiple stages. The number of stages and criteria associated at each stages can vary based on organizational culture and the type of ideation. We recommend a three-stage process that has worked well for most of our communities. The criteria for moving ideas forward in the initial stage should be fairly simple such as buzz, conversational levels, votes, etc. The second stage should raise the bar on simple metrics and add more in depth scrutiny (e.g. Multidimensional reviews, expert evaluations, etc.). The last stage should deploy a mechanism like idea markets that leads the users reveal their true preferences.

Idea and user recommendations
Every social innovation community I have examined demonstrates the long tail behavior in terms user activity levels as well as the number of users that collaborate on a single idea. There is typically a very small percentage of users that are extremely active and a few ideas that attract widespread attention. On the other hand a large number of users participate at a more moderate level and  there are many ideas that attract only a handful of collaborators. Interestingly, the ideation process is very democratic. Successful ideas don't just come from hyperactive users, many of them come from the long tail. As a result enabling collaborators to find each other on the long tail becomes crucial to success.

As the idea volume goes up, it becomes increasingly difficult for the long tail users to connect with like minded users and ideas that they have an interest in. In such cases, the innovation platform must make or recommend such connections based on user profiles, their participatory behavior, and social network analysis.The connections can be presented via a user-centric presentation of ideas and like-minded users. Alternatively recommendations can be pushed to the users via email notifications.

Shortening attention spans, multitasking, and the rise of Twitter

Number of theories abound regarding why Twitter has become so popular including this one that claims that it is because it is designed to have social intent. I started thinking about this and in particular thought about why I am not hooked on to Facebook but find Twitter useful. I came away with three reasons for Twitter's success: overabundance of information, shortening of attention spans, and the desire to create personal brands. Twitter thrives in this environment because it offers the most efficient way of discovering pertinent information via social curating.

The problem of information explosion predates the rise of the Internet in the consumer space; it just got worse with Internet's ubiquity and ease of access. A number of mechanisms have evolved in the Internet age to help improve information discovery as shown in the figure below. Note that the new mechanisms haven't replaced the older ones but simply expanded available modes of discovery.

Release of Netscape browser was the first sign of impending rise of the Internet. I remember getting excited about this cool new browser that was much better than Mosaic. Netscape was a great improvement over its predecessor but you pretty much had to type in the URL of a Web site until Yet Another Hierarchical Officious Oracle (Yahoo) came along and started "organizing" the Web into a hierarchy of categories. Now you could traverse the categories, find a topic of interest and browse available web sites pre-classified to be in that category. The next step in the evolution was Google with its PageRank based technology that was very effective in adding relevance to search. This provided a much better mechanism to find information in which "reputable" Web sites related to search keywords were listed at the top.

When Web 2.0 arrived on the scene, we saw the emergence of folksonomies (tag clouds) and social bookmarking/tagging sites like Delicious, StumbleUpon, and Digg. These sites gently introduced Web readers to the notion of social curating when they started following bookmarks, posts, etc. created by people they trusted. Twitter perfected this art by providing the most efficient way of filtering information via social curating. Each Twitter user follows other users in the"circle of trust" that includes his friends, colleagues, and experts in the areas of interest. Resulting twitter feed then provides a much better way of finding interesting and useful content since it is either published or recommended by the people you trust.

Twitter is also very efficient in the way it gets information to you. Since the tweets are limited to 160 characters, it forces publishers to be succinct in their messages, a valuable gift for the recipients in the age of shortening attention spans. In general we are all getting used to and expecting instant gratification. Five day cricket test matches have for the most part been replaced by 20-20 games that last for about three hours; average movie lengths have been descending; younger generation talks in short bursts that almost sounds like an audio Twitter feed.

In a sense a Twitter feed is like the front page of a newspaper. The front page displays introductory portion of a number of news articles, most of the remaining article is on other pages. With Twitter, you get only the headlines and if you find that interesting, you click on a link that takes you to another place that shows you all the details.

The third reason for Twitter's popularity is that it is the most efficient way of advertising your personal brand. With Web 2.0 came a wave of new individuals that finally had the opportunity to create and maintain a personal brand. A number of now-famous individuals rode the blogging wave to make themselves into famous journalists, political analysts, critiques, reviewers, and advisers. The blogging revolution created a tournament (read more about the tournament behavior in Freakonomics) of information hawkers in which people compete to get rich in the information economy. Most competitors become part of the long tail and eventually stop playing. Some make it big and live on. From the information publishing viewpoint, Twitter  offers a very efficient mechanism to advertise your offering. It is also very effective since information consumers increasingly rely on these headlines to get to the content they ultimately want to see.

Although Twitter founders must be quite happy with its success, the underlying trends that make it popular are a bit disturbing. It's rise is accompanied by addiction to passive information consumption, shortening attention spans, and increased multitasking. Addiction to Twitter is probably best explained by the theory of unpredictable rewards (the same rationale that explains email addiction): you get rewarded with really interesting piece of information at a random frequency, so you keep reading hoping to find that next reward.

Twitter joins the crowded ranks of other competing sources of information like email, television, etc. that are fragmenting attention spans. These "distractions" coupled with the increased pace and complexity of modern life are forcing multi-tasking behavior and frequent context switching. My sense is that these conditions are seriously affecting the ability to focus on and adequately examine important tasks. Some of the research in this area cited in this New York Times article concludes that multi-taskers are worse at filtering out irrelevant information and switch tasks. You can even test yourself on these abilities here.

Since going back to stone ages is not a particularly good solution, the only option is to adapt to the new reality and find ways to minimize its impact on productivity. To that end I leave you with the following techniques I personally employ that allows me to be a player an not just a spectator:

• Shop when you need something, not when there is a sale - This works with information as well. Instead of continuous looking for information, I choose the most effective way of information retrieval that helps me with the job at hand.
• Play only during recess - I reserve my random explorations (twitter feed browsing, searches on innovation, etc.) for selected time periods
• Publish less - I am not on the frequent Twitter plan. For the most part, I try to tweet when I think it is of broad relevance to reduce information pollution.
• Avoid reciprocal follower-ship - I am a bit choosy about who to follow leaving me with relatively less number of tweets to browse through

A word from innovation professionals

I had never thought of myself as an "Innovation Professional" but I heard the term during the Process Driven Innovation conference I recently attended in Philadelphia and realized that I fall into that category or at least deal with that bunch every day. There was off course much more to learn besides the existence of aforementioned term. This was a small conference with around 50-60 people in attendance, very similar in size and scope to the one I attended a few months ago in Ibiza. Great thing about such smaller conferences is that they give you an opportunity to have in-depth exchange of ideas with people that are actively involved in innovation management (aka innovation professionals).

A lot of what I heard during the conference is common wisdom shared by the community. For example many speakers emphasized the need to have top down support, a continual communication strategy, obstacles created by the nay sayers, etc. I was on a panel that discussed such best practices along with Jon Bidwell (Chubb), Kevin Paylow (Halliburton), and Steve Fenessey (InnoCentive). I did have a few surprises and heard some interesting perspectives.

The Rise of "The Innovation Professional"
Innovation is undoubtedly the hottest buzzword these days. For example, you will find vastly increased number of people with the word "innovation" in their titles or job function if you search on Linked In today. That's great news, however, the innovation owners very often do not have the budget or the authority to implement ideas; that is the prerogative of the P&L owners. I have found this dichotomy very frustrating since it requires a dance between two different owners to achieve your innovation objectives.

My view was that innovation should really be a part of everyone's job function and we need to eliminate this dichotomy. Contrary to my belief, I heard overriding sentiment in the conference that the specialized innovation role is a must-have in an organization. This view sounds self-serving at first but as I heard people talk about their experiences I found an answer to this dilemma. No one was disputing the fact that every employee, customer, and partner should be part of the innovation process. They were simply pointing out that innovation teams are instrumental in orchestrating that process. They play a role very similar to the one played by brokers, advertisers, distributors in the free market economy.While everyone chips in with their ideas and sponsorships, innovation team administers the process, helps people present their ideas, and connects different players to optimize the process.

Have idea management tools become a commodity?
The speaker from 3M made a comment that the ideation tools have become a commodity. We certainly seem to hear about a new ideation tool every day that has a catchy interface and a cool slogen that claims to help you eliminate world hunger. I agree with this assertion if you limit the scope of idea management to idea collection, commenting, and voting. That has certainly been commoditized and almost all ideation tools would work very well for smaller communities with campaign style ideation. Unfortunately if you select a platform that is limited to this functionality only, it does not scale very well for large communities.

Innovation is not an event, it is an ongoing process. Making this process successful requires a combination of active community management, education, and the right tool set. Many routine tasks become major pain points as the community size increases. Spammers and users that try to game rankings and rewards becomes a bigger issue. Sponsors find it increasingly difficult to read each and every idea posted in the system. They need assistance in finding common themes and duplicate ideas. My feeling is that social innovation (or for that matter socializing any business process) is in it's infancy. Some of the tools in the market offer a far better alternative to older methods but there is still much to be learned. Until that happens, it is hard to imagine this space becoming commoditized.

Prelude to Idea Collection
Can Campbell Soups learn anything from Nike? Seems like the soups folks were inspired by the way Nike is improving customer loyalty by creating a Web site that provides fitness help. What does cat litter have in common with burn bandages? Both need absorbent material that removes odors. These are some examples of how businesses can innovate by observing other companies in acompletely different sector or products that seem unrelated but have solved analogous problems. Such strategies are crucial for a successful ideation process. In fact idea sourcing or collection itself is the easiest phase compared to what comes before (ideation framing and creative thinking methodologies) and after (evolution, evaluation, selection, and implementation).

Whirlpool presentation described another strategy to spawn creative thinking. Traditionally the decision to buy appliances is made by the woman of the house and appliances go either in the kitchen or the laundry room. When you present these observations, somewhat obvious questions that come to your mind are: why target only women? and what about other rooms in the house? This lead to the Gladiator brand that caters to men and focuses on solutions for organizing the clutter in the garage.

A couple of presenters including the person from Campbell Soups described observation-focused ways of innovating. Both of these projects, however, involved a small number of people in the analysis of those observations. I think they will see much better results if they allowed a much larger number of people to examine those observations. In other words combining traditional methodologies with social innovation tools like Spigit would provide a huge benefit.

Miscellenous Tidbits

• Innovation is a Sale - Really? - Jean-Mark Frangos from British Telecom took the innovation professionalism to yet another level by casting his job function as making a sale. In this analogy, ideas are products, P&L owners are the customers, articulation and visualization are your sales tools. Now that's innovative
• Andrew Douglass from Rhodia illustrated how six-sigma techniques can be applied to the innovation process. Their existing process was resulting in a high failure rate for ideas that had passed the proof of concept stage. By examine the causes of failure and preparing a check list of criteria at the early stage reduced the failure rate substantially.
• Heard about the thank you program in IBM where employees can suggest gifts for other employees based on their personal experience. I thought this would be a good addition to Spigit's virtual economy

Social networks and Dunbar's number

In response to my earlier post about Metcalfe's Law, Lior Sion brought up another magic number that is frequently cited when it comes to the number of social relationships: Dunbar's Number. I remembered reading about this in Malcolm Gladwell's Tipping Point and it got me thinking about how it applies to online social networks.

To get started in my investigation, I looked up the Wikipedia article on Dunbar's number. The article describes the number as "a theoretical cognitive limit to the number of people with whom one can maintain stable social relationships". As I continued my search I found a better reference, Dunbar himself explaining how he arrived at this number in this video . According to Dunbar, the number of meaningful relationships a primate can maintain depends on the ratio of the size of neocortex (which exists only in mammals) to the overall brain size. You can compare this ratio and corresponding social unit size in different primates and arrive at a number of roughly 150 in humans. There are several examples of social units without a significant hierarchy that support this hypothesis including Goretex micro-businesses cited in Gladwell's book and Hutterite communes.

So how is this number related to online social communities? Apparently Dunbar is investigating this issue as reported in this article. The full report is due some time later this year. Not surprisingly he notes that even in case of people who have millions of "friends", the communication frequencies indicate that the number of meaningful connections is less than 150. I ran some numbers on Spigit's communities and I came away with the same conclusion. The chart below shows the total number of connections as a function of individual users for three community sizes: small (< 500), medium (< 5000) and large (> 5000). I cut off the long tail in order to clearly show the variation in the fat part (please note that no animals where harmed in this process).

The most conspicuous thing about the charts is the fact that the number of connections as a function of users seems to follow the power law. An interesting property of power law functions is that the natural log of frequencies (y values) varies linearly with log of x values. I did exactly that to the values shown in the first chart and plotted the resulting values in the chart below.

The top few highly connected users fail to live up to the power law expectations (Wikipedia article mentions similar deviations in other phenomena that otherwise follow power law), but the shape of the curve is quite close to being linear towards the end.

Looking at the first chart it is quite clear that the bulk of users have less than 150 connections but what about the highly talkative users that seem to have communicated over a thousand other individuals? The answer lies in the fact that none of these users maintained all these conversations simulteneously. Online communities differ significantly from  the social groups studied earlier in three inter-related respects:

1. Lower barrier to entry - Internet is highly efficient communication mechanism. It is incredibly easy to get virtual access to people that you have no prior knowledge of. 
2. Higher mobility - It is much easier to leave or join a social cluster compared to other social units where physical proximity plays a critical role.
3. Driven by Individual Goals - Online clusters form around a common interest or a goal that is initiated at the individual level not at the communal level. Offline social units form around a set of predefined common goals (e.g. Gore factory unit, HOAs, etc.). Social units (certainly within Spigit communities) in online communities are spawned by people who want to change the world around them.

These three factors contribute to a very dynamic set of social clusters that are formed around topics of interest, gain traction with a small group of users and are eventually replaced by other hot spots with a different membership. In fact looking under the covers, it was clear to me that even highly social users changed their group affiliations over a period of time. Even though they remained interested in communicating with a handful of individuals on a continuous basis, they communicated with different sets of people at different times. The chart shown below illustrates this point. It shows the communication pattern of some of the most connected users in highly successful Spigit communities. Each data point reflects the number of connections in a 30 day period. As you can see from the chart, that number is below 150 with the exception of one data point even though the total number connections over the entire period is above 1000 for some of these users.

So why is this analysis so interesting (other than satisfying one's academic curiosity)? Well, because it helps us evaluate the breadth of collaboration in an online community which is strongly correlated with diversity of participation.  The analysis presented here and in my last post about Metcalfe's law helps us set expectations in this regard. First, the number of interactions per user follows the power law even in the most successful communities. In general the best you can hope to achieve is to lift the long tail  part of the curve upwards. Second, whether it is due to cognitive or bandwidth limitations, the number of active communications in a short time frame that can be maintained by even the most active individuals seems to be below 150. As a result it is not realistic to judge the level of collaboration in terms of the total number of possible connections (n*(n-1)/2), but Dunbar's number seems to be an achievable ceiling for the most active users in your community.

Living up to Metcalf's law on social innovation communities

In my presentation last week at spigit's customer summit, I presented some statistics on spigit's social innovation communities including a comparison between actual connections established over a social network and the maximum possible connections - the basis for Metcalf's law (which is really not a law but we will continue the tradition for the time being). Given the warm reception from the audience at my explanation, I thought I will dig into this a little bit more. This blog post is the result of my investigation.

First let me set some context on Metcalfe's Law. Metcalfe, inventor of the Ethernet protocol, was originally talking about how the "value" of a network of compatibly communicating machines grows quadratically (not linearly). I found this post that shows a reproduction of his original slide. The basis for this assertion is the fact that in a network of "n" nodes, each node can communicate with n-1 nodes hence there are n*(n-1)/2 possible pair-wise connections on such a network. The value of a network is really in its power of establishing communication among its nodes. The number of possible connections is of the order O(n²) as n becomes large which leads to Metcalfe's law.

The mathematics works the same for any network so it is possible to apply the basic thought to other networks including human ones. If each person in an on-line community communicate with all other members, the total number of communication links will be of the order O(n²). Although this may be the ideal scenario, what's the reality like? I performed some analysis on Spigit communities to answer that question. The chart below shows the number of connections in the network plotted against the size of the social network. Three types of connections are plotted:

1. Conversational - Post-response connections between original poster of an idea, blog post, forum thread etc. and a commenter. Note that multiple exchange between two people count as one connection.
2. Conversations and Evaluations - In addition to conversations, this includes voting/rating interactions between two users
3. All connections - These include viewer-poster and linked-in style connections in addition to 1&2

This sanitized version of the graph does not show the numerical data but does depict the trends I observed in my analysis. The chart also shows a plot of linear and nlog(n) functions to give readers an idea about the order of change. Looking at the chart and the numbers behind it, it is clear that the number of conversational connections increases more or less linearly with the community size. The number of connections representing all interactions certainly did not grow at a quadratic rate but the fluctuations were too high to draw any meaningful conclusion. The linear variation in conversational connections is consistent with the long tail pattern of conversations found in all communities. There are a relatively fewer number of individuals who are extremely well connected, but the majority of users are part of the long tail and have a limited number of connections that is independent of the community size.

It is evident from the chart that there is a significant variation in the connectivity utilization between different communities. When I examined the dynamics within more successful communities (in terms of user connections), I found three key factors:

1. Well-defined innovation process -  All successful communities have a very well defined process consisting of idea sourcing, evaluation, and selection. The process is well publicized and executed as promised.
2. Incentives - Most communities have fine-grained incentives for encouraging broader participation in addition to allocating funding for implementing selected ideas. The incentives range from tangible items like gift cards to recognition by leadership to purely monetary rewards corresponding to the play money earned in Spigit's virtual economy.
3. Emphasis on "social" - Some of the well connected communities emphasize social interactions by promoting blogging, open exchange of thoughts on discussion forums, and (now) traditional collaboration over wikis. Innovation is embedded within this larger context.

It is interesting to note that not every successful community incorporated all three factors. Also in general increased connectivity did not necessarily lead to a better result in terms of number of quality ideas that get selected for implementation. That is strongly correlated only to the first two factors. Conversely in some communities,  organizational culture inhibited people from freely collaborating on proposed ideas, but the outcome has been significantly better compared to traditional processes due to support from the executive team and strong innovation process management. This is an important point to consider when setting up enterprise communities. Whereas communication within consumer-oriented social networks is very often an end in itself, for enterprise communities communication is a means to an end. For spigit communities, the goal is the selection and implementation of ideas that provide business value and achieving that requires channeling community interactions to that end.

It's also worth mentioning a few patterns that I thought I would see but didn't. I didn't find any correlation between different business sectors and network connectivity nor did I find more interactions within communities that had higher percentage of knowledge workers. There was C-level involvement in a couple of cases but in many communities the innovation effort was lead by an internal champion who was passionate about making the innovation effort successful. Finally the size of the community didn't seem to have much effect on the networking pattern even though it did have an effect on the number of participants that posted on the innovation site.