Yoda: Real-Time Crowd-Sourcing + Twitter Meta-RECAP

0Shares

yoda with light saber — Got Crowd? BE the Force!

How to Perfect Real-Time Crowdsourcing

The new techniques behind instant crowdsourcing makes human intelligence available on demand for the first time.

One of the great goals of computer science is to embed human-like intelligence in common applications like image processing, robotic control and so on. Until recently the focus has been to develop an artificial intelligence that can do these jobs.

But there's another option: using real humans via some kind of crowdsourcing process. One well known example involves the CAPTCHA test which can identify humans from machines by asking them to identify words so badly distorted that automated systems cannot read them.

However, spammers are known to farm out these tasks to humans via crowdsourcing systems that pay in the region of 0.5 cents per 1000 words solved.

Might not a similar process work for legitimate tasks such as building human intelligence into real world applications?

The problem, of course, is latency. Nobody wants to sit around for 20 minutes while a worker with the skills to steer your robotic waiter is crowdsourced from the other side of the world.

So how quickly can a crowd be put into action.?That's the question tackled today by Michael Bernstein at the Massachusetts Institute of Technology in Cambridge and a few pals.

In the past, these guys have found ways to bring a crowd to bear in about two seconds. That's quick. But the reaction time is limited to how quickly a worker responds to an alert.

Now these guys say they've find a way to reduce the reaction time to 500 milliseconds–that's effectively realtime. A system with a half second latency could turn crowdsourcing into a very different kind of resource.

The idea that Bernstein and co have come up with is straightforward. These guys simply “precruit” a crowd and keep them on standby until a task becomes available. Effectively, they're paying workers a retainer so that they are available immediately when needed

The difficulty is in the messy details of precruitment. How many workers do you need to keep on retainer, how do cope with drop outs and how do you keep people interested so that they are available to work at a fraction of a second's notice?

Bernstein and co have used an idea called queuing theory to work out how to optimise the process of precruitment according to how often the task comes up, how long it takes and so on.

They've also developed an interesting psychological trick to keep workers ready for action. When workers are precruited, a screen opens up on their computer which downloads the task. The download occurs extremely quickly but if no task is to hand, the screen shows a “loading” bar.

It turns out that the loading bar keeps workers focused on the forthcoming task for up to ten seconds, at which point their attention begins to wander. At that point, if no task materialises, the worker can be paid off.

Bernstein and co have even tested how well this works using a whack-a-mole type task which appears on workers screens after a randomly chosen period between 0 and 20 seconds. They recruited 50 workers to carry out 373 whacks and found the median length of time between the mole's appearance and the worker moving the mouse toward the mole to click on it was 0.50 seconds.

“Our model suggests…that crowds could be recruited effectively instantaneously,” they say.

That could change the nature of crowdsourcing. Bernstein suggest that real time crowdsourcing could be used to point cameras, control robots and produce instant opinion polls.

But first crowdsourcers will have to change the way they do business. Bernstein and co suggest that they could build a retainer into their system design so that they have a pool of ready-to-go workers available at any instant.

Workers could even be assessed at how good they are for this kind of task allowing them to build up a reputation for the work. Bernstein and co suggest two new reputation statistics–the percentage of time workers respond to a precruitment request and how quickly they respond on these occasions.

Shouldn't be too hard to set up. An interesting new business model for the likes of Mechanical Turk and others or perhaps for an enterprising new start up.

Reference: Analytic Methods for Optimising Real time Crowdsourcing

See Also:

$250 Device Global Maps & Text Messages

$169 SPOT Connect: Any Cell Phone SatComm Capable

2012 PREPRINT FOR COMMENT: The Craft of Intelligence

2012 Integrity, Reflexivity, & Open Everything

2012 THE OPEN SOURCE EVERYTHING MANIFESTO: Transparency, Truth & Trust

2010 INTELLIGENCE FOR EARTH: Clarity, Diversity, Integrity, & Sustainability

2010 The Ultimate Hack Re-Inventing Intelligence to Re-Engineer Earth (Chapter for Counter-Terrorism Book Out of Denmark)

2010: Human Intelligence (HUMINT) Trilogy Updated

Crowdsourcing Mapped Water Data via SMS/txt msg

Earth Intelligence Network