Posts Tagged ‘crowdsourcing’
What does Charlton Heston and crowdsourcing have in common?
It’s PEOPLE!
Seems Microsoft is attempting to “integrate human expertise permanently into our writing tools” with a plug in that leverages crowdsourced labor via Mechanical Turk to create a better spell/grammar checker.
Mechanical Turk is an on-demand workforce platform that leverages the crowd (anyone, anywhere, who’s interested and connected to the Internet) to compete routine, time-consuming tasks that are difficult for computers but easy for humans – commonly referred to as human intelligence tasks or HITs.
Turkers – as the workers on the site are called – are paid nominal fees to complete HITs. Often one group may complete a task and another verifies quality of others.
How does it work?
Soylent is an add-on that leverages Mechanical Turk to copy-edit your document. Currently in Beta, Soylent attempts to “embed human knowledge into a word processor.”
Soylent uses a program design pattern called “Find-Fix-Verify” that splits task into smaller tasks that can be done in stages. Theory is that this decreases costs but increased quality.
Features include:
- Shortn – Turkers cut out extra words and shorten your manuscript
- Crowdproof – leverages the crowd to check spelling grammar and provide suggestions about style
- The Human Macro – allows you to describe the types of changes you want (e.g., change all to past tense), then turn it loose to the crowd
Ask not what the you can do for the people, but what the people can do for you.
While Clippy came with Microsoft Word, Soylent requires payment to crowdworkers.
Costs are descried as “small” or “just a few cents” so it’s hard to really estimate real costs. The creators say it costs about $1.50 per paragraph.
Are you game?
Check out the short YouTube video.
You can join the Beta to see what the people can do for you. If you do, let me know what you think.
Instead of turning to paid analysts, or internal experts some companies are using the crowd to help improve the accuracy of demand forecasts as well as better manage inventory and manufacturing capacity.
Predictive markets, often also referred to as information markets, aggregate the knowledge of the crowd to make predictions regarding unknown future events. By aggregating distributed knowledge the predictions of the crowd are often more accurate than when companies rely on only a handful of experts.
How do they work?
In prediction markets, individuals buy and sell “futures” or “shares” based on their beliefs regarding the probability of the event taking place. If they are correct, they are rewarded for their efforts. Because of U.S. laws related to online gambling, rewards often take the form of play money, gift certificates, or recognition within the market.
What is interesting about prediction markets is their structure creates incentives for individuals to act on their closely held information. Because rewards are tied to correct predictions, individuals in the crowd who have access to more information, which may aid in their understanding of the market, tend to buy more shares than those who are just guessing.
For example, Google uses over 300 internal prediction markets to assess events such as customer demand for new products (“How many Gmail users will there be on January 1, 2009?”), company and product performance (“When will the first Android phone hit the market?”), and competitor performance (“How many iPhones will Apple sell in the first year?”). In addition to new sources of predictions, Google has used these prediction markets to better understand and improve the flow of information within their company.
Do predictive markets work?
As with many new uses of the crowd, we have only scant evidence regarding the effectiveness of prediction markets. Best Buy reports internal prediction markets designed to predict holiday sales of gift cards were 99.5% accurate compared to a 95% accuracy rate from external consultants. Intel also reports success with their internal “forecasting markets.”
Faced with the difficult task of predicting demand of products requiring lead times of months or even quarters, Intel found their internal markets were at least as accurate as official figures and in some cases more accurate by 20% (i.e., 20% less error).
There is, however, evidence that outside factors can impact results of these markets. Some data suggests employees may be overly optimistic regarding company performance tending to artificially inflate positive results. For publically traded companies, stock performance can influence predictions upward or downward in line with the latest market swings. And finally, there are also numerous issues related to providing the right incentives for participation as well as obtaining executive buy-in for such initiatives.
Often the term crowdsourcing is associated with large groups of people contributing information or data (e.g., Wikipedia), with product innovation (e.g., Dell’s IdeaStorm), or with advertising (e.g., Doritos and Pepsi superbowl ad). But a number of companies are also leveraging the crowd to better serve customers and reduce those long wait times.
One area where the crowd is being leveraged is to supplement or even replace call centers. Because companies often struggle with issues related to ensuring operators have the right level of expertise to efficiently and effectively answer incoming questions, some researchers are suggesting that turning to active user communities as a source of expertise may be a more efficient and cost effective way of providing continuous and expert-level support. Active community members often represent the most knowledgeable customers and are likely already providing advice as part of their online activities. Even if only a small percent of calls can be re-directed to these über-users, it could result in substantial cost reductions and potentially more satisfied customers.
Companies such as HP, Microsoft, and AT&T are currently leveraging their user communities to supplements call center staff. In fact, Intuit reports a reduction in total support calls for TurboTax during tax season by 40%. Other companies such as giffgaff, a UK mobile phone operator, leverage their user community forums to handle 100% of their customer support issues, most within 5 minutes. In addition to leveraging their own communities, companies are working with intermediaries who connect them with other knowledge communities. For example, FixYa.com is an online service that “leases” access to knowledgeable crowds to help companies supplement their current customer support services. And Arise leverages 120,000 highly trained, home based independent contractors to provide high quality support resulting in a 25-30% cost savings relative to traditional brick-and-mortar call centers.
Of course, there are many potential issues with leveraging the crowd for customer service. How do you motivate über-users to act as on-call experts? How can you ensure that someone in the community can answer the question in a timely fashion, after all they don’t work for you? And what are the risks of having non-employees act in a capacity that suggests a company sanctioned response? Certainly these are difficult questions.
So the next time you call a customer service line, you might strike up a conversation and see who’s actually answering the phone.
Crowdsourcing #1 for 2012
Posted on: August 2, 2011
As a researcher studying crowdsourcing I was excited to see that Haydn Shaughnessy of Forbes magazine predicts that crowdsourcing will be top of mind for companies in 2012. While I agree that crowdsourcing examples are on the rise, I’m not sure I agree that crowdsourcing is a “fail safe” option that is a “mature” as Haydn suggests.
We’ve only begun to examine the economic impacts of crowdsourcing initiatives on the corporate bottom line. Some studies are finding that turning to the crowd has reduced cost and time for product innovation and problem solving, improved quality, and increased market acceptance of new products. In fact, TopCoder a site that runs contests for developing complex software applciations reports that projects typically requiring over a year of development have been completed in slightly over five months. Additionally, TopCoder programs average .98 bugs per thousand lines of code, significantly better than the industry standard of six per thousand. These initial findings are promising, but more research is needed to determine the true benefits to corporations.
While potentially more economical than traditional innovation methods, crowdsourcing does not come without costs. It is not a “build it and they will come” solution. Success requires defined business goals, an understanding of crowd dynamics as well as collaborative technologies. Additionally, those who are getting the crowd to participate are often finding it difficult to sort through and evaluate all the information and ideas that are generated.
One of the biggest hurdles is organizational culture. I saw a similar issue when working with companies to leverage social media for marketing initiatives. Success at leveraging the crowd requires an organizational culture that embraces open methods from the top down and is willing to give up some control. Exposing yourself and your company to the crowd can be scary and isn’t without risk. Lawyers raise concerns about leakage of trade secrets and issues related to intellectual property. Employees may feel they are becoming obsolete and fear for their jobs. And, executives may pull the plug when they encounter negative feedback or comments from customers.
Every day there are new and different uses of the crowd for innovation. While companies like P&G and intermediaries like InnoCentive seem to have it down, most are only beginning to experiment with leveraging the crowd for innovation. I do agree that crowdsourcing may be an excellent opportunity for companies to supplement or even replace their current innovation initiatives – saving money and time in the process. But currently we have only scant evidence of the how best companies can extra value from the crowd.
(Cartoon (c) Geek and Poke, 2009)
Part of the reason I decided to go back to get my PhD was to study how social media was changing how businesses connect with customers and build brands. I’m currently focusing on “crowdsourcing.” Crowdsourcing is basically an open call to the “crowd” to participate in an activity typically completed by employees or paid consultant/contractor.
There are tons of different types of crowdsourcing sites and researchers are only beginning to examine the different uses of the crowd. I wrote a post about the crowdsourcing site eBird a while back. Here’s another example of a crowdsourced site. This one for crowdsourced software development.
TopCoder.com connects companies with programmers in the crowd to collectively build complex programs.
Here’s how it works:
- Clients specify requirements, timelines, and budgets and the crowd competes to see who can produce the best code in the allotted timeframe.
- Qualified reviewers evaluate weekly submissions, scores are posted for everyone to see, and a winner is selected.
- After all modules are complete, a new contest is held to assemble the modules into the final program.
- Winners are paid a pre-defined fee and coders in second place receive half the amount of the winner.
- Winners turn over code (and all rights to it) to the paying company.
You’d think that throwing out a programming challenge to an undefined group of people without set standards or guidelines would result in pretty “iffy” code. But, what’s interesting is that TopCoder code actually exceeds the industry standard for quality. TopCoder reports an average of .98 errors per 1000 lines of code, compared to the industry average of 6 per 1000.
TopCoder manages to create complex programs in less time, at less cost, and at a higher quality than typical of internal development teams. Is this the end of internal software development teams?
Crowdsourcing, for the birds.
Posted on: March 21, 2010
No, I don’t mean crowdsourcing is “for the birds,” as in worthless. I literally mean crowdsourcing for birds. Here’s an interesting application of turning to the crowd to generate data – bird data.
Cornell Lab of Ornithology and the National Audubon Society have joined together to leverage the collective productivity of the crowd to accomplish a task they would never be able to do on their own. They have created the online site eBird.org.
eBird enlists the help of bird watchers across North America to collect and document the presence of specific bird species. Using the eBird site, birders can report sightings and access the latest data on bird populations across the western hemisphere.
By leveraging the interests of the crowd and providing the ability for them to easily contribute to a shared cause, site sponsors are now able to accurately track bird populations more quickly and economically than they could with in-house staff. In 2006, more than 4.3 million observations were submitted by bird watchers across the country.
Interestingly, the site doesn’t have any collaborative tools where birders can talk to each other, but it doesn’t appear to need them to generate a sense of community.
If you know of other sites that leverage the crowd to collect data, please let me know.
Wired Pursuits 