Yoda: Real-Time Crowd-Sourcing + Twitter Meta-RECAP

Advanced Cyber/IO, Collective Intelligence
Got Crowd? BE the Force!

How to Perfect Real-Time Crowdsourcing

The new techniques behind instant crowdsourcing makes human intelligence available on demand for the first time.

One of the great goals of computer science is to embed human-like intelligence in common applications like image processing, robotic control and so on. Until recently the focus has been to develop an artificial intelligence that can do these jobs.

But there’s another option: using real humans via some kind of crowdsourcing process. One well known example involves the CAPTCHA test which can identify humans from machines by asking them to identify words so badly distorted that automated systems cannot read them.

However, spammers are known to farm out these tasks to humans via crowdsourcing systems that pay in the region of 0.5 cents per 1000 words solved.

Might not a similar process work for legitimate tasks such as building human intelligence into real world applications?

The problem, of course, is latency. Nobody wants to sit around for 20 minutes while a worker with the skills to steer your robotic waiter is crowdsourced from the other side of the world.

So how quickly can a crowd be put into action.?That’s the question tackled today by Michael Bernstein at the Massachusetts Institute of Technology in Cambridge and a few pals.

In the past, these guys have found ways to bring a crowd to bear in about two seconds. That’s quick. But the reaction time is limited to how quickly a worker responds to an alert.

Now these guys say they’ve find a way to reduce the reaction time to 500 milliseconds–that’s effectively realtime. A system with a half second latency could turn crowdsourcing into a very different kind of resource.

The idea that Bernstein and co have come up with is straightforward. These guys simply “precruit” a crowd and keep them on standby until a task becomes available. Effectively, they’re paying workers a retainer so that they are available immediately when needed

The difficulty is in the messy details of precruitment. How many workers do you need to keep on retainer, how do cope with drop outs  and how do you keep people interested so that they are available to work at a fraction of a second’s notice?

Bernstein and co have used an idea called queuing theory to work out how to optimise the process of precruitment according to how often the task comes up, how long it takes and so on.

They’ve also developed an interesting psychological trick to keep workers ready for action. When workers are precruited, a screen opens up on their computer which downloads the task. The download occurs extremely quickly but if no task is to hand, the screen shows a “loading” bar.

It turns out that the loading bar keeps workers focused on the forthcoming task for up to ten seconds, at which point their attention begins to wander. At that point, if no task materialises, the worker can be paid off.

Bernstein and co have even tested how well this works using a whack-a-mole type task which appears on workers screens after a randomly chosen period between 0 and 20 seconds. They recruited 50 workers to carry out 373 whacks and found the median length of time between the mole’s appearance and the worker moving the mouse toward the mole to click on it was 0.50 seconds.

“Our model suggests…that crowds could be recruited eff ectively instantaneously,” they say.

That could change the nature of crowdsourcing. Bernstein suggest that real time crowdsourcing could be used to point cameras, control robots and produce instant opinion polls.

But first crowdsourcers will have to change the way they do business. Bernstein and co suggest that they could build a retainer into their system design so that they have a pool of ready-to-go workers available at any instant.

Workers could even be assessed at how good they are for this kind of task allowing them to build up a reputation for the work. Bernstein and co suggest two new reputation statistics–the percentage of time workers respond to a precruitment request and how quickly they respond on these occasions.

Shouldn’t be too hard to set up. An interesting new business model for the likes of Mechanical Turk and others or perhaps for an enterprising new start up.

Reference: Analytic Methods for Optimising Real time Crowdsourcing

See Also:

$250 Device Global Maps & Text Messages

$169 SPOT Connect: Any Cell Phone SatComm Capable

2012 PREPRINT FOR COMMENT: The Craft of Intelligence

2012 Integrity, Reflexivity, & Open Everything

2012 THE OPEN SOURCE EVERYTHING MANIFESTO: Transparency, Truth & Trust

2010 INTELLIGENCE FOR EARTH: Clarity, Diversity, Integrity, & Sustainability

2010 The Ultimate Hack Re-Inventing Intelligence to Re-Engineer Earth (Chapter for Counter-Terrorism Book Out of Denmark)

2010: Human Intelligence (HUMINT) Trilogy Updated

Crowdsourcing Mapped Water Data via SMS/txt msg

Earth Intelligence Network

Future of Multinational Intelligence & Operations

Graphic: Twitter as an Intelligence Tool

Howard Rheingold: Crap Detection & Critical Thinking

Howard Rheingold: Infotention Skills + Citizen Intel RECAP

Howard Rheingold: News Filters for the Future – Technical Services or Human Networks?

Howard Rheingold: Open Source Intelligence Meets Real-Time News and Data Curation – SwiftRiver

Journal: DARPA & MIT Discover “Share the Wealth”

Journal: Haiti Public Intelligence Emergent

Journal: Haiti–Twitter Rocks

Journal: IC on Twitter, Still Not Making Sense

Journal: Neither Twitter nor Facebook Lead Revolution

Journal: One Mobile Per Child

Journal: Twitter Aggregation Way Cool

Journal: Twitter Breaks the News

Journal: Whither Twitter?

Koko: Crowd-Sourcing Weather Forecasting

Letter to Secretary General of the United Nations

Letter to Sir Richard Branson “The Virgin Truth”

Letter to Microsoft CTO Operation Cloudburst

Letter to Pope Benedict XVI

M4IS2 (Multinational, Multiagency, Multidisciplinary, Multidomain Information-Sharing and Sense-Making)

Letter to President Obama ON INTELLIGENCE

Mini-Me: When All Else Fails, Try Crowd-Sourcing

Multinational Information-Sharing

Patrick Meier: Crisis Mapping Syria – Automated Data Mining and Crowdsourced Human Intelligence

Peter Meier: Crowdsouring Syrian Imagery & Crisis Reports

Patrick Meier: Filtering Tweets Real-Time for Crisis Response

Patrick Meier: How to Crowdsource Better Governance in Authoritarian States

Patrick Meier: Mobile Technologies, Crisis Mapping, & Disaster Response

Patrick Meier: The Digital Operations Center of the American Red Cross

Patrick Meier: Trails of Digital Trustworthness

Patrick Meier: Ushahidi Emergent as Democracy in Being

Paul Fernhout: Open Letter to the Intelligence Advanced Programs Research Agency (IARPA)

Reference: Lee Felsenstein & Dave Warner Converse

Reference: Strategic Analytic Model for Creating a Prosperous World at Peace

Review: A Democratic Approach to Sustainable Futures — A Workbook for Addressing the Global Problematique

Review: The World Is Open–How Web Technology Is Revolutionizing Education

Review: The World Sensorium — The Social Embryology of World Federation 1946

Search: global brain human brain + RECAP

Search: holistic cybernetic integral

Search: whole earth models

Steele on World Brain to Amazon Developers Conference (VIDEO 2007)

Steele on Open Everything to Gnomedex (VIDEO 2007)

Steele on Open Source Future at Echo Chamber (AUDIO 2006)

Tom Atlee: New Developments in Co-Intelligence Work

Whitehouse Needing Online and Mobile Crowdsourcing & Collective Intelligence for Decision-Support

Worth a Look: CrowdMap (Beta)

World Brain & Global Game 101-104

Worth a Look: Momentum Crowdsourcing for Open Source Intelligence

Yoda: Big Data Tough Love, Everyone Fails