Living up to Metcalf's law on social innovation communities

In my presentation last week at spigit's customer summit, I presented some statistics on spigit's social innovation communities including a comparison between actual connections established over a social network and the maximum possible connections - the basis for Metcalf's law (which is really not a law but we will continue the tradition for the time being). Given the warm reception from the audience at my explanation, I thought I will dig into this a little bit more. This blog post is the result of my investigation.

First let me set some context on Metcalfe's Law. Metcalfe, inventor of the Ethernet protocol, was originally talking about how the "value" of a network of compatibly communicating machines grows quadratically (not linearly). I found this post that shows a reproduction of his original slide. The basis for this assertion is the fact that in a network of "n" nodes, each node can communicate with n-1 nodes hence there are n*(n-1)/2 possible pair-wise connections on such a network. The value of a network is really in its power of establishing communication among its nodes. The number of possible connections is of the order O(n²) as n becomes large which leads to Metcalfe's law.

The mathematics works the same for any network so it is possible to apply the basic thought to other networks including human ones. If each person in an on-line community communicate with all other members, the total number of communication links will be of the order O(n²). Although this may be the ideal scenario, what's the reality like? I performed some analysis on Spigit communities to answer that question. The chart below shows the number of connections in the network plotted against the size of the social network. Three types of connections are plotted:

1. Conversational - Post-response connections between original poster of an idea, blog post, forum thread etc. and a commenter. Note that multiple exchange between two people count as one connection.
2. Conversations and Evaluations - In addition to conversations, this includes voting/rating interactions between two users
3. All connections - These include viewer-poster and linked-in style connections in addition to 1&2

This sanitized version of the graph does not show the numerical data but does depict the trends I observed in my analysis. The chart also shows a plot of linear and nlog(n) functions to give readers an idea about the order of change. Looking at the chart and the numbers behind it, it is clear that the number of conversational connections increases more or less linearly with the community size. The number of connections representing all interactions certainly did not grow at a quadratic rate but the fluctuations were too high to draw any meaningful conclusion. The linear variation in conversational connections is consistent with the long tail pattern of conversations found in all communities. There are a relatively fewer number of individuals who are extremely well connected, but the majority of users are part of the long tail and have a limited number of connections that is independent of the community size.

It is evident from the chart that there is a significant variation in the connectivity utilization between different communities. When I examined the dynamics within more successful communities (in terms of user connections), I found three key factors:

1. Well-defined innovation process -  All successful communities have a very well defined process consisting of idea sourcing, evaluation, and selection. The process is well publicized and executed as promised.
2. Incentives - Most communities have fine-grained incentives for encouraging broader participation in addition to allocating funding for implementing selected ideas. The incentives range from tangible items like gift cards to recognition by leadership to purely monetary rewards corresponding to the play money earned in Spigit's virtual economy.
3. Emphasis on "social" - Some of the well connected communities emphasize social interactions by promoting blogging, open exchange of thoughts on discussion forums, and (now) traditional collaboration over wikis. Innovation is embedded within this larger context.

It is interesting to note that not every successful community incorporated all three factors. Also in general increased connectivity did not necessarily lead to a better result in terms of number of quality ideas that get selected for implementation. That is strongly correlated only to the first two factors. Conversely in some communities,  organizational culture inhibited people from freely collaborating on proposed ideas, but the outcome has been significantly better compared to traditional processes due to support from the executive team and strong innovation process management. This is an important point to consider when setting up enterprise communities. Whereas communication within consumer-oriented social networks is very often an end in itself, for enterprise communities communication is a means to an end. For spigit communities, the goal is the selection and implementation of ideas that provide business value and achieving that requires channeling community interactions to that end.

It's also worth mentioning a few patterns that I thought I would see but didn't. I didn't find any correlation between different business sectors and network connectivity nor did I find more interactions within communities that had higher percentage of knowledge workers. There was C-level involvement in a couple of cases but in many communities the innovation effort was lead by an internal champion who was passionate about making the innovation effort successful. Finally the size of the community didn't seem to have much effect on the networking pattern even though it did have an effect on the number of participants that posted on the innovation site.

Designing idea markets for social innovation communities

Cisco's I-Prize competition hosted by Spigit concluded recently. I thoroughly enjoyed seeing lots of interesting ideas coming through and the amount of collaboration that followed. From Cisco's viewpoint the great success story was the billion dollar idea that was selected at the end of the competition. For me, the most successful outcome was the fact that our idea market worked! The idea selected by Cisco had the maximum amount of investment in the market and the overall ranking matched up very well with the internal evaluation process.

I am a big proponent of deploying Idea Markets within social innovation communities. Some of our key clients have made that core to their idea ranking and selection process and have seen excellent results in terms of the quality of ideas that bubble up to the top. In this blog post I explain the main concept behind idea markets, how to use them in practice and why I think they work better than some of the other techniques for ranking and rating ideas.

Role of Idea Markets in Social Innovation
One of the core features a social innovation platform is its ability to leverage the crowd for evaluation and ranking of ideas. There are many ideation tools that attempt to do this based on simple voting measures. In the simplest case, ranking is done based on popularity, the idea with the most "Thumbs Up" votes wins. This approach is appealing for its simplicity but is prone to gaming. The gaming behavior can be significantly reduced by introducing user reputation in idea ranking computation. Instead of simply counting up votes or ratio of positive to negative votes (approval rating), ranking can be computed as a reputation weighted average of votes. We have seen this work very well in ideation communities especially in the initial stage of the idea lifecycle. The approach still has its limitations. Very often frivolous ideas (e.g. 20% raise for all employees) get to the top even though there is no chance of them getting accepted for implementation. Voting is almost too easy to perform. On the positive side this leads to greater participation, on the negative side it promotes careless or even worse cliquish voting behaviors.

Idea markets, if designed in the right way, can overcome these shortcomings. Spigit uses its virtual economy as a basis for an incremental reward system for users. Idea market is one of the most interesting aspects of this economy. We usually introduce idea trading after filtering out the noise via conventional means. Idea market is then used to determine idea ranking in the final stage. The market is set up in such a way that users see a pay-off on their investment only if the idea gets selected. When the currency is tied to a reward system, users are motivated to invest only in those ideas with a higher chance of getting implemented. As a result idea markets more effectively suppress adverse behaviors like gaming, careless selections, voted-for-my-friend, etc.

Market Design Principles
Simplicity
Idea Markets work well only if the markets are thick and diverse. Simplicity of the investment model is probably the most critical factor that determines participation levels. For example, "short selling" may be understood very well by Wall Street traders, but is quite confusing for the average user and will most likely inhibit participation.

Incentives Tied to Success Potential
The ultimate objective of running idea markets is to recognize the best ideas based on the crowd's input. The "stock" price of an idea determines the ranking and therefore must be tied directly to its potential for success. Correspondingly the payoff model must incent users to invest in ideas based on their eventual chance of success and suppress day trading and cliquish behaviors that work against the main goal of identifying promising ideas.

Early Investment equals Bigger ROI
The design should provide better rewards to those who invest early and should decrease as the idea gets more popular. In general this encourages users to spread out there investments in many ideas and inhibits "jumping on the band wagon" behavior.

Embedded Collaboration
While prediction market traders are investing in outcomes they cannot control, idea markets are about trading in ideas that can be greatly enhanced or be allowed to fail fast based on inputs from market participants. Idea markets therefore should provide users the ability to express their views, suggest modifications, and supply additional information.


Independent Idea Selection
Idea markets should have an independent mechanism for selecting successful ideas and should reward investors in those ideas. Without independent verification, markets are much more prone to gaming since being on the top of the stock market fully guarantees a reward. Many users then choose to invest for guaranteed rewards albeit for a lesser amount.

Idea Market Models
The main concept behind idea markets is simple. Market participants are given play money when they join. They may earn more money by posting ideas, voting, commenting, etc. The market administrator allows participants to invest their money in some or all ideas posted on the system. The "stock" price (although it is not visible to the user in the variation that I like) of an idea goes up with the number of investors and/or amount of investment. I have seen and experimented with three different idea markets that implement the basic concept described in the last paragraph. These are described below.

Stock Markets with Continuous Double Auction
Traditionally idea markets have simulated real world financial markets to a large extent. Each idea is traded as a stock market instrument. Ideas typically open at a fixed price (typically $10 with idea price varying from $1-$99). The markets run using the Continuous Double Auction (CDA) mechanism in which traders place bids and asks at a certain price and share quantity. The system executes a trade when it finds a bid at an equal or higher price than an ask. Markets either allow short selling or award a fixed number shares to all traders in order to inject liquidity in the market. The final closing price is based either on the last executed trade or more often Volume Weighted Average Price (VWAP) over the last few days preceeding the close. Users are awarded prizes based on their portfolio value

Stock Markets with a Automated MarketMaker
I have seen automated market maker implementations in prediction markets but not within idea markets except the one offered by Spigit.  We have used the stock market paradigm combined with a system market maker on Spigit platform with a fair amount of success.  In this model, the market maker sets the idea price based on the level of investment and number of investors in that idea relative to all other ideas in the market place. Users trade with the market maker when they want to buy or sell shares in an idea.

Investment-Based Market
In this market implementation, users bet or invest money in ideas they like. They don't buy or sell stock, but simply indicate the amount of money they want to invest in an idea. A user can adjust their investment at any time before an idea is closed. Users that invest in successful ideas make profit on their investment. The virtual ROI is conveyed to the user at the time of the investment. Early investments often yield higher profits and changing investments does not yield any profits.

Market Comparison
We have deployed all three types of market for our clients. The collaborative aspect can be embedded in all three models. An independent mechanism for selecting successful ideas can be established for all models as well. The models do differ with respect to the other three considerations that I have laid out earlier.

The stock market like models are hard to understand for most users outside of the financial sector and companies with highly educated workforce. Some companies do prefer the stock market paradigm due to the fun factor they bring in. They both reward early investors but encourage more active users in purely profit-seeking transactions that go against their own evaluation of ideas.

The pure investment-based model is simple to understand for most people. The only way to earn profit is to bet on the right idea and the ROI diminishes as the idea gets more popular with investors. We have successfully used this model for Cisco I-Prize and some of the internal communities. That's certainly my preference but would love to hear your thoughts!