Introducing Social Lair

In my last blog post, I described some of the lessons I have learned in creating and deploying social innovation communities. I have created a new platform that builds upon the lessons learned for our new company Social Lair (http://sociallair.com). In this blog post I want to outline the solution strategy adopted by Social Lair that really "puts social to work".

Getting work done is at the core of all our offerings. A good enterprise social platform will get the work done faster, better, and cheaper by  reaping the benefits of collective wisdom of crowds. We believe that every business process, from innovation and idea management to HR, Product Management, can benefit from becoming more open and democratic. In fact, the enterprise social software will become mission-critical only when we achieve this goal.

At the conceptual level, our solution can be viewed in terms of three main layers (depicted in the figure below):

• Blended Social Analytics - Analytics is the core foundation of our solution stack and consists of Information Network Analysis (reputations, connector rating, conversational levels), Natural Language Processing (purpose-based routing, expert identification), Quantitative Aggregation Models (prediction markets, rank aggregation), and Gamification (currency management). Analytics piece is instrumental in optimizing cost of social transactions and adding relevance to search and ranking.
• Purposeful Conversations – We model community interactions as a series of structured conversations focused on a business purpose (market forecasting, choice resolution, ideation, etc). Each conversation type has an unstructured component, a quantitative component, and an aggregation and analytics engine for providing actionable intelligence to the conversation sponsor.
• Community Archetypes – Based on our past experience, we have realized that different business contexts require different styles of community design. For example, Idea campaign communities would just not work effectively for daily collaboration.  At the same time, we did not want to create a system that is TOO flexible and requires greater knowledge and effort for set up and configuration. Our approach is to capture well-established collaboration patterns in the form of community archetype. Each archetype is defined in terms of the included conversation types, navigational scheme, terminology, and access control.

Our strategy in realizing this vision is to create minimalistic experience that reflects our approach and iterate on it based on the feedback received by our end users. We are already having some interesting conversations with end users in this space who are getting quite excited about the potential of our technology. I will off course keep you updated in the new venture as we continue our journey in this space.

Lessons learned in developing social innovation communities

About five years ago, I started building a social innovation platform that has now been successfully deployed by several Fortune 500 companies. These communities have yielded great results: new products, improved process efficiency, greater customer satisfaction; just to name a few. Those successes were also accompanied by realization that these communities are extremely hard to sustain. Ironically, the two most important factors were top leadership support and a passionate individual/team whose sole job was to run innovation (Sorry Gamification!). Moreover, this individual or the team needed to constantly care and feed the community to keep it going.
I have also been watching a slew of “enterprise social” software companies entering the market. We tried to use some of these products ourselves and came to the same realization I had five years ago: socializing without purpose isn’t going to stick; it ends up producing more noise driving away serious users.
As I digested this information and observations, I came up with a few things that have to be done differently to address the adoption malaise in the enterprise social arena:
Focus on “Collaboration” not “Socialization” – Business leaders, quite justifiably, frown upon employees using Facebook or Twitter at work. Simply bringing these technologies inside your organization will not be productive. Although there is a need to facilitate undirected, water cooler conversations, you have to put “business” before “social” to create “business social” solutions.
Create a “Funnel of Funnels” – The “Idea Funnel” model of user engagement is easy to understand and works for one or two campaign but ultimately requires enormous effort from the single sponsor to keep it going. The engagement platform needs to make it easy for a broader cross section of participants to create their own funnels. Campaign style communities democratize idea generation part, but fail to do the same to funnel creation.
Judicious Use of Gamification – Applying game design techniques to improve user engagement levels is tricky. Unlike game designers, enterprise community managers are interested, not in raw user activity, but in meaningful participation that produces results consistent with sponsor objectives. I have seen virtual currency awards leading to spamming behavior by users simply trying to earn that $10 gift card. Conversely, I have seen instances of self-motivated community members contribute great ideas without needing leaderboards or badges.
More Efficient Social Transactions – The ROI associated with enterprise social deployments comes from two sources: cost savings and revenues associated with business transactions, and efficiencies in establishing “social transactions”. The former is well understood and can be easily quantified, so I won’t go into that here. A social transaction is an interaction between two or more stakeholders associated with a business objective. An effective enterprise social platform would minimize the time and costs associated with social transactions, especially in case of weak or non-existent ties within transacting parties.

Now that's what you call "innovation"!

I don't usually engage in propagating oddball email chains but I couldn't resist publishing these images recently forwarded to me by a friend of mine. These are not so called "breakthrough" innovations, but clearly represent out of the box thinking and a value for the inventors :-)

Theft Deterrent Device

Wine Cooler

Kitchen Accessories

Hands Free Calling for California Drivers

Recycle, Reuse, Reduce

What's the similarity between crowdsourcing and IBM's DeepQA design strategies

I attended TieCon 2011 conference last week and got to hear a fascinating keynote given by Dr. David Ferrucci on IBM's DeepQA project. IBM's "Watson" recently beat human competitors in Jeopardy, a prospect much harder than beating a human at Chess since it involves reasoning over unstructured information. 

The lecture had its lighter moments including this incident narrated by Dr. Ferrucci that illustrates how social networking has become ingrained in the psyche of the younger generation. Apparently he asked a jeopardy question to a bunch of middle-schoolers as part of his presentation. The question under the category "Licoln Logs" was:

"Treasury Secy. Chase just submitted this to me for the third time - guess what pal this time I am accepting it." The answer off course is "resignation". 

Answer from the seventh grader?  

"A friend Request".

Other than such incidents and some of the hilarious answers given by Watson in early stages of development, there were a couple of slides that really caught my attention. The first one was about what drove the DeepQA team to success in this challenging task. Three simple things:

1. Irresistible Vision - A vision of the future you know in your gut must happen.
2. A bit of wisdom - It can be now.
3. A lot of mettle - It will be us.

The second interesting part of the presentation was the core set of strategies adopted in this work:

• Rejection of complex hard coded models - The team did not rely on hand-coded ontologies representing formal models about the world. They found them to be too narrow, slow, and really not up to the challenge.
• Intelligence from many shallow methods - The decision of whether to answer a question and what that answer would be was derived by combining multiple reasoning methods many of them shallow.
• Massive parallelism - Original version of Watson would take a couple of hours to come up with a decision and it needed to be within 3 seconds. Answer: parallel explorations of the answer space

If you think about these strategies, you will realize that these are pretty much the reasons for solving problems through social input a.k.a crowd-sourcing. Experts (hard coded knowledge representations) must realize they don't know everything. Amalgamation of many view points spanning the entire information space produces better results (provided certain conditions are met) that individual data points. Turning loose thousands of minds on a problem gets it solved much quicker.

My three principles of new product development

Spigit recently released a Facebook application that exposes Spigit's social innovation management functionality on the platform. It is always thrilling when something you help create sees the light of the day. In this case it was particularly satisfying because we followed my three principles of new product development with a great degree of success (thanks to FacebookEngage product manager Nina Chai and Pratim Mukharjee, the lead developer on the project).

Simple but not Simpler
I have lead the evolution of the Spigit platform from an abstract concept to the leading social innovation platform used by over 100 organizations all over the world. Initially we had to take a leap of faith in creating the first version of the product. Combining idea management with social networking was a new concept for a lot of our prospective users. We certainly made a lot of untested assumptions to create that first version. We did strive, however, to minimize the number of assumptions and create something that was simple (with a minimal feature set) but not simpler, i.e. it provided unprecedented benefit to the end users. We have started Spigit's Facebook application development much the same way. The initial version exposes minimal functionality. It's a proposal to our customer base which they use, review, and evaluate.The rest of the course will be charted by combining customer feedback with our own intuition and understanding of the marketplace.

Native Experience
There are some products that are so revolutionary that they change existing interaction paradigms, processes, or even the way of thinking. Most others need to seamlessly fit into existing  lifestyles and usage conventions. They must blend with established usage patterns and feel completely native to the user. We had to be very careful about this blend for the facebook application. A large portion of Spigit's user base is in the enterprise space. Facebook users are significantly different in their expectations, attention spans, and motivation to participate. Spigit's Facebook application deliberately sports a non-enterprise, lightweight look. The primary motivation of Facebook users is keep in touch with their social circle. The Spigit apps plays into that by making the user aware of ideas posted by your friends and how many of them made into the top ranking list.The ideas are displayed very much in a wall-style format. When a user posts an idea, posting that on the wall or sending message can be done with minimal effort. In general, we have made every effort to introduce Spigit functionality in a way that is consistent with ethos of Facebook community.

The Wow Factor
A new product should have at least one or two features that make a user go "Wow that's useful!" when s/he uses the product. When I started using iPhone, I had that reaction when I realized that unlike my previous mobile windows-based phone, the screen goes dark when I put it to my ear but it comes alive when I hold it in front of me. With Spigit's facebook launch we did not have a problem introducing the Wow factor. We pulled in some of Spigit's key features deemed relevant to Facebook. We included idea theme clusters that allows administrators to get a sense of emerging trends and topics. Reputation scores identify key contributors in the community. Spigit's social analytics is available to the administrators via Spigit's regular user interface. Subsequent versions of the app will track user feedback and will build upon the initial feature set to make it even better.

Thoughts on spigit, idea management, and future of social innovation

Javed Mohammad, a community manager at Spigit, interviewed me recently. This blog post simply the transcript of that interview. The conversation has also been posted on the Spigit blog.

JM Before companies like Spigit came along how was idea management handled?


PD Before spigit, idea management was mainly done in a very top-down manner. A very small set of technology and business leaders were responsible for innovation. The flow of ideas was mainly uni-directional, going from the leadership to the rest of the organization.

JM And what was the implication of that?

PD The process was very rigid and lacked transparency. We have now seen several instances where so called experts make decisions that turn out to be worse than those made based on input from a diverse set of people. The universe of ideas was also vastly smaller since the process relied on a very small group of people.

JM Somewhere along the line you got the inspiration for Spigit. What was it?

PD At the time we founded Spigit, social networking was already popular in the consumer space. We had seen isolated instances of blogs and wikis that went beyond just the social chatter and attempted to provide business value. For example, we were noticing a lot of bloggers proposing interesting ideas with business potential. The original concept was to create a product that would facilitate turning these raw ideas into real products and services.

JM How did you combine what would seem to be two incompatible applications one consumer and the other business into a product like Spigit?

PD That's really our key distinction. When we started Spigit, you either had consumer-facing applications that supported online social networks or you had traditional enterprise applications that supported very opaque and rigid processes like Stage Gate. We wanted to retain and leverage the social, fun aspect of consumer oriented applications, but at the same time channel that social energy in a very productive way. Secondly, we needed an engine for analyzing social exchanges and create actionable intelligence for our enterprise sponsors.

JM How does one provide “fun” in an Enterprise application?

PD Introducing "fun" in enterprise applications is getting very popular these days. In face now there is a name for it: "Gamification". Gamification is the application of game design principles to non-game applications to influence user behavior in accomplishing real life tasks. It is about creating a fun, engaging experience for users that encourages them to perform tasks that they might otherwise find tedious or common place. Frequent flyer programs is a classic example of how gamification helps build loyalty. In spigit, we realize it via leaderboards, user badges, reputation, idea markets, etc.

JM We hear a lot about “the wisdom of crowds.” Who or which type of organizations have been able to leverage this wisdom?

PD Any organization where there is a common goal and collaboration of its community. They have to be open, transparent and able to execute. As long as they are willing to implement ideas that come through the idea funnel, and the community sees that as a reinforcing behavior, those are the organizations that thrive.

JM Can you give any examples that are public?

PD AT&T has a company-wide innovation program built on top of the spigit platform. That's an example of an internal idea management program that is very democratic. Cisco's I-Prize that was looking for the next billion dollar idea is an example of external facing application of Spigit's technology. I-Prize is also an excellent example of the power of idea markets, one of the key gamification features in spigit. The top three ideas predicted by the markets were exactly the same as the ones selected by Cisco.
JM Gartner in their Hype cycle for emerging technologies shows Idea Management as having crossed the trough of disillusionment. When do you think it will reach the plateau of productivity which I guess is Nirvana?

PD Idea management, the way Spigit pioneered it, began about 4 years ago. I have seen a measurable change in the attitude of enterprise decision makers towards this technology. Questions have changed from "should we" to "which tool". I think it would still take one or two more years to reach maturation and it may not reach the plateau in isolation. Idea Management will most likely be combined with other business processes like Talent and Resource Management, CRM, etc.

JM What excites you about the field of social innovation and what’s your prediction for the future of ideation?

PD The current solutions have done a good job of decentralizing idea collection process. We could definitely improve collaboration and idea evolution by providing intelligent matchmaking between potential collaborators. In many cases, ideation framing is still done by a central group. There is still a single sponsor and they are responsible for doing selection and implementation. That part needs to be decentralized. Fundamentally, it is all about collaborative problem solving and the future should enable a decentralized, peer-to-peer communication among seekers and solvers.

JM Thank you.

Essential features of a scalable social innovation platform

The overall outlook and expectations towards social business software have changed dramatically over the last few years. Most enterprise decision makers have stopped questioning the value of social technology and are now more concerned with finding the right enterprise social platform that can support large communities within and beyond organizational boundaries. The trend is especially stronger in open innovation (or social innovation as I like to call it) communities. As a result, a number of existing players in innovation management have re-focused their product strategy to incorporate social elements. We are also seeing a large number of new idea management products entering the market at a rapid pace.

Although it seems that idea and innovation management tools have become a commodity, the reality is that most tools are limited to running campaign style idea collection leaving idea evaluation, filtering, and ranking in the hands of a small group of sponsors. This approach may be sufficient for some of the smaller ideation campaigns but does not scale very well when the community size and the number of ideation efforts go up. In order to be scalable, a social innovation platform must enable sponsors and community managers to:

• Clearly frame ideation objectives and engage the right audience for optimal results
• Manage online behavior of community members such that meaningful contributions are promoted and spamming/gaming attempts are minimized
• Ensure diversity of participation and connect the right set of collaborators belonging to the long tail
• Create an engaging experience that keeps users coming back
• Leverage the crowd to filter and rank ideas to minimize bias and management overhead
• Choose the right balance between crowd sourcing and top-down control

In this blog post I outline a core of set of features that must be present in a social innovation platform in order to effectively meet the needs listed above. Note that I focus only on advanced features that go beyond basic functionality such as idea posting, commenting, and voting.

Multi-tenant architecture and templates
Innovation efforts in large organizations involve multiple regions, lines of businesses, and ideation goals. We certainly recommend creating a global ideation site that allows the entire community to collaborate across organizational silos, however it is also necessary to create destinations visited by different population segments. The social innovation platform must allow sponsors to easily set up new communities and invite relevant sections of the user population for participation. Moreover it must allow innovation leaders to set up community templates with terminology, look and feel, evaluation and ranking mechanisms representing different ideation objectives and policies. Community templates significantly reduce the barrier to entry for new ideation sponsors; accelerates the learning curve and thereby minimizes effort required by the core innovation team to bring the new entrants up to speed.

User reputation
Innovation communities face two opposite types of problems: users posting too many ideas or an inactive community with too few ideas. The root cause in most cases is improper ideation framing. Some large communities, however, suffer from the former problem in spite of having a dedicated innovation team that clearly defines the goals. When the noise drowns out the substance, it makes it more difficult to make productive contributions and drives away serious users. Computing user reputation is an effective strategy to cope with this problem.

It is important to note that reputation is a reflection of quality of contribution not quantity. Some idea management tools use the term reputation to indicate activity level. Reputation rank should be determined not by how many times a user posts on the community, but how he community reacts to the user's online behavior and participation.

The reputation ranking of users has three implications for community management. First, displaying reputation on the leaderboard is a way of giving social recognition to your top contributors. It also creates a sense of competition, a great motivator for almost every kind of user. Second, the reputation score can be factored in idea ranking greatly improving relevance. Third, if reputation is tied to currency or points earned by users, it encourages users to be active in a meaningful way.

Reputation can also be used as a stick. The innovation platform must detect spammers and users who attempt to game the system. Reputation score for such users should be dropped to zero or a very low number thereby limiting their ability to get points and influencing idea ranking.

Game mechanics
The term game mechanics or "Gamification" is becoming quite popular in the context of social media discussions. This obviously does not translate to introducing Farmville to your community. That will probably result in having the "too few ideas" problem that I mentioned earlier. Gamification should really be about creating a fun, competitive environment for your innovation community that makes the site sticky in a positive manner. It should be designed in such a way that winning the game requires participants to align their behaviors with innovation objectives and strive to improve overall community health.

Gamification goes much beyond getting Foursquare badges (although that could definitely be part of it). It should include leaderboards that displays users by reputation, virtual wealth, and top innovators in addition to different types of idea rankings. Users should be able to acquire special powers that allows them to moderate content, view restricted areas or access functionality not available to other users. Mechanisms like idea markets and prediction markets are another example of constructive gamification. These mechanisms are not only a fun way of engaging the crowd, but they also enable sponsors to aggregate opinions of many and derive results that assist in decision making and making predictions.

Automated idea graduation
I recently heard a great quote about the value of crowdsourcing:

It's not difficult to find a needle in the haystack if you enlist the haystack to find it.

The point here is that it's a lot easier to find great ideas and solutions when you open up ideation challenges and problems to the entire community. I believe in going even further and enlisting the crowd in filtering and ranking of ideas in addition to initial suggestions. When you leave idea evaluation and ranking entirely in the hands of the sponsors, it is problematic in a couple ways. First, you really expose yourself to the biases shared by a small group of people. The HiPPO problem (highly paid persons opinion) is a corollary of this issue. Second, the innovation committee becomes a bottleneck in the innovation process as the number of ideas increase.

The innovation platform must support a way of filtering ideas in multiple stages. The number of stages and criteria associated at each stages can vary based on organizational culture and the type of ideation. We recommend a three-stage process that has worked well for most of our communities. The criteria for moving ideas forward in the initial stage should be fairly simple such as buzz, conversational levels, votes, etc. The second stage should raise the bar on simple metrics and add more in depth scrutiny (e.g. Multidimensional reviews, expert evaluations, etc.). The last stage should deploy a mechanism like idea markets that leads the users reveal their true preferences.

Idea and user recommendations
Every social innovation community I have examined demonstrates the long tail behavior in terms user activity levels as well as the number of users that collaborate on a single idea. There is typically a very small percentage of users that are extremely active and a few ideas that attract widespread attention. On the other hand a large number of users participate at a more moderate level and  there are many ideas that attract only a handful of collaborators. Interestingly, the ideation process is very democratic. Successful ideas don't just come from hyperactive users, many of them come from the long tail. As a result enabling collaborators to find each other on the long tail becomes crucial to success.

As the idea volume goes up, it becomes increasingly difficult for the long tail users to connect with like minded users and ideas that they have an interest in. In such cases, the innovation platform must make or recommend such connections based on user profiles, their participatory behavior, and social network analysis.The connections can be presented via a user-centric presentation of ideas and like-minded users. Alternatively recommendations can be pushed to the users via email notifications.