Thursday, May 19, 2011

What's the similarity between crowdsourcing and IBM's DeepQA design strategies

I attended TieCon 2011 conference last week and got to hear a fascinating keynote given by Dr. David Ferrucci on IBM's DeepQA project. IBM's "Watson" recently beat human competitors in Jeopardy, a prospect much harder than beating a human at Chess since it involves reasoning over unstructured information. 

The lecture had its lighter moments including this incident narrated by Dr. Ferrucci that illustrates how social networking has become ingrained in the psyche of the younger generation. Apparently he asked a jeopardy question to a bunch of middle-schoolers as part of his presentation. The question under the category "Licoln Logs" was:
"Treasury Secy. Chase just submitted this to me for the third time - guess what pal this time I am accepting it." The answer off course is "resignation". 

Answer from the seventh grader?  
"A friend Request".

Other than such incidents and some of the hilarious answers given by Watson in early stages of development, there were a couple of slides that really caught my attention. The first one was about what drove the DeepQA team to success in this challenging task. Three simple things:
  1. Irresistible Vision - A vision of the future you know in your gut must happen.
  2. A bit of wisdom - It can be now.
  3. A lot of mettle - It will be us.
The second interesting part of the presentation was the core set of strategies adopted in this work:
  • Rejection of complex hard coded models - The team did not rely on hand-coded ontologies representing formal models about the world. They found them to be too narrow, slow, and really not up to the challenge.
  • Intelligence from many shallow methods - The decision of whether to answer a question and what that answer would be was derived by combining multiple reasoning methods many of them shallow.
  • Massive parallelism - Original version of Watson would take a couple of hours to come up with a decision and it needed to be within 3 seconds. Answer: parallel explorations of the answer space
If you think about these strategies, you will realize that these are pretty much the reasons for solving problems through social input a.k.a crowd-sourcing. Experts (hard coded knowledge representations) must realize they don't know everything. Amalgamation of many view points spanning the entire information space produces better results (provided certain conditions are met) that individual data points. Turning loose thousands of minds on a problem gets it solved much quicker.