Patrick Meier: Using Twitter to Detect Micro-Crises in Real-Time

Crowd-Sourcing, Geospatial
Patrick Meier
Patrick Meier

Using Twitter to Detect Micro-Crises in Real-Time

Social media is increasingly used to communicate during major crises. But what about small-scale incidents such as a car crash or fire? These “micro-crises” typically generate a far smaller volume of social media activity during a much shorter period and more bounded geographical area. Detecting these small-scale events thus poses an important challenge for the field of Crisis Computing.

Click on Image to Enlarge
Click on Image to Enlarge

Axel Schulz just published co-authored a paper on this exact challenge. In this study, he and co-authors Petar Ristoski & Heiko Paulheim ”present a solution for a real-time identifi cation of small scale incidents using microblogs,” which uses machine learning—combining text classi cation and semantic enrichment of microblogs—to increase situational awareness. The study draws on 7.5 million tweets posted in the city centers of Seattle and Memphis during November & December 2012 and February 2013. The authors used the “Seattle Real Time Fire 911 Calls” dataset to identify relevant keywords in the collected tweets. They also used WordNet to “extend this set by adding the direct hyponyms. For instance, the keyword “accident” was extended with ‘collision’, ‘crash’, ‘wreck’, ‘injury’, ‘fatal accident’, and ‘casualty’.”

An evaluation of this combined “text classi cation” and “semantic enrichment” approach shows that small scale incidents can be identified with an accuracy 89%. A copy of Axel et al.‘s paper is available here (PDF). This is a remarkable level of accuracy given the rare and micro-level nature of the incidents studied.

Read rest of post.

Patrick Meier: Using Big Data to Inform Poverty Reduction Strategies — Data Science for Social Good: Not Cognitive Surplus but Cognitive Mismatch

01 Poverty, 03 Economy, 07 Health, 11 Society, Crowd-Sourcing, Geospatial
Patrick Meier
Patrick Meier

Using Big Data to Inform Poverty Reduction Strategies

My colleagues and I at QCRI are spearheading a new experimental Research and Development (R&D) project with the United Nations Development Program (UNDP) team in Cairo, Egypt. Colleagues at Harvard University, MIT and UC Berkeley have also joined the R&D efforts as full-fledged partners. The research question: can an analysis of Twitter traffic in Egypt tell us anything about changes in unemployment and poverty levels? This question was formulated with UNDP’s Cairo-based Team during several conversations I had with them in early 2013.

Read full post.

Data Science for Social Good: Not Cognitive Surplus but Cognitive Mismatch

I’ve spent the past 12 months working with top notch data scientists at QCRI et al. The following may thus be biased: I think QCRI got it right. They strive to balance their commitment to positive social change with their primary mission of becoming a world class institute for advanced computing research. The two are not mutually exclusive. What it takes is a dedicated position, like the one created for me at QCRI. It is high time that other research institutes, academic programs and international computing conferences create comparable focal points to catalyze data science for social good.

Read full post.

Michelle Monk: Geke.US Lays Out Government-Corporate Circles of Corruption

Access, Crowd-Sourcing, Data, Design, Economics/True Cost, Education, Geospatial, Governance, Innovation, Knowledge, Money, Politics, Transparency
Michelle Monk
Michelle Monk

Here is what they have as of today.  An extraordinary effort that should soon become automated.   You won't find these on LinkedIn!  Click for individual Venn diagram similar to the Keystone Pipeline shown below.,

Nominally Good:

AFL-CIO

The Rest:

Comcast   .   Defense Contractors   .   Disney   .   Enron   .   Fannie Mae   .   General Electric   .   Goldman Sachs   .   Green Energy   .   Keystone Pipeline   .   Media   .   Monsanto   .   Motion Picture Association of America   .   Oil Industry   .   Pharmaceuticals   .   Planned Parenthood   .   Social Networking Sites   .   Tobacco Industry   .   Walmart

Click on Image to Enlarge
Click on Image to Enlarge

Phi Beta Iota: Now imagine this being done at every level of government from the municipality on up, across every policy domain, with whole systems analytics, true cost economics, and all trade-offs clearly visible. This is where we are going.  Humanitarian technology and Open Source Everything (OSE) are going to empower the public in a manner no government or corporation can conceive or achieve.

Patrick Meier: Using Waze, Uber, AirBnB and SeeClickFix for Disaster Response

Crowd-Sourcing, Design, Geospatial, Governance, Software
Patrick Meier
Patrick Meier

Using Waze, Uber, AirBnB and SeeClickFix for Disaster Response

After the Category 5 Tornado in Oklahoma, map editors at Waze used the service to route drivers around the damage. While Uber increased their car service fares during Hurricane Sandy, they could have modified their App to encourage the shared use of Uber cars to fill unused seats. This would have taken some work,  but AirBnB did modify their platform overnight to let over 1,400 kindhearted New Yorkers offer free housing to victims of the hurricane. SeeClick fix was used also to report over 800 issues in just 24 hours after Sandy made landfall. These included reports on the precise location of power outages, flooding, downed trees, downed electric lines, and other storm damage. Following the Boston Marathon Bombing, SeeClick fix was used to quickly find emergency housing for those affected by the tragedy.

Read full post with additional graphics and links.

Continue reading “Patrick Meier: Using Waze, Uber, AirBnB and SeeClickFix for Disaster Response”

Patrick Meier: The Geography of Twitter: Mapping the Global Heartbeat

Crowd-Sourcing, Geospatial
Patrick Meier
Patrick Meier

The Geography of Twitter: Mapping the Global Heartbeat

My colleague Kalev Leetaru recently co-authored this comprehensive study on the various sources and accuracies of geographic information on Twitter. This is the first detailed study of its kind. The detailed analysis, which runs some 50-pages long, has important implications vis-a-vis the use of social media in emergency management and humanitarian response. Should you not have the time to analyze the comprehensive study, this blog post highlights the most important and relevant findings.

Read full post with global graphics.

Patrick Meier: Could CrowdOptic Be Used For Disaster Response?

Crowd-Sourcing, Geospatial
Patrick Meier
Patrick Meier

Could CrowdOptic Be Used For Disaster Response?

Crowds—rather than sole individuals—are increasingly bearing witness to disasters large and small. Instagram users, for example, snapped 800,000 #Sandy pictures during the hurricane last year. One way to make sense of this vast volume and velocity of multimedia content—Big Data—during disasters is with PhotoSynth, as blogged here. Another perhaps more sophisticated approach would be to use CrowdOptic, which automatically zeros in on the specific location that eyewitnesses are looking at when using their smartphones to take pictures or recording videos.

“Once a crowd’s point of focus is determined, any content generated by that point of focus is automatically authenticated, and a relative significance is assigned based on CrowdOptic’s focal data attributes […].” These include: (1) Number of Viewers; (2) Location of Focus; (3) Distance to Epicenter; (4) Cluster Timestamp, Duration; and (5) Cluster Creation, Dissipation Speed.” CrowdOptic can also be used on live streams and archival images & videos. Once a cluster is identified, the best images/videos pointing to this cluster are automatically selected.

Read full post with graphics and more links.

Patrick Meier: Data Mining Wikipedia in Real Time for Disaster Response [or Any Current Event]

Crowd-Sourcing, Data, Geospatial, Governance, Innovation, P2P / Panarchy, Resilience
Patrick Meier
Patrick Meier

Data Mining Wikipedia in Real Time for Disaster Response

My colleague Fernando Diaz has continued working on an interesting Wikipedia project since he first discussed the idea with me last year. Since Wikipedia is increasingly used to crowdsource live reports on breaking news such as sudden-onset humanitarian crisis and disasters, why not mine these pages for structured information relevant to humanitarian response professionals?

In computing-speak, Sequential Update Summarization is a task that generates useful, new and timely sentence-length updates about a developing event such as a disaster. In contrast, Value Tracking tracks the value of important event-related attributes such as fatalities and financial impact. Fernando and his colleagues will be using both approaches to mine and analyze Wikipedia pages in real time. Other attributes worth tracking include injuries, number of displaced individuals, infrastructure damage and perhaps disease outbreaks. Pictures of the disaster uploaded to a given Wikipedia page may also be of interest to humanitarians, along with meta-data such as the number of edits made to a page per minute or hour and the number of unique editors.

Click on Image to Enlarge
Click on Image to Enlarge

Fernando and his colleagues have recently launched this tech challenge to apply these two advanced computing techniques to disaster response based on crowdsourced Wikipedia articles. The challenge is part of the Text Retrieval Conference (TREC), which is being held in Maryland this November. As part of this applied research and prototyping challenge, Fernando et al. plan to use the resulting summarization and value tracking from Wikipedia to verify related  crisis information shared on social media. Needless to say, I’m really excited about the potential. So Fernando and I are exploring ways to ensure that the results of this challenge are appropriately transferred to the humanitarian community. Stay tuned for updates. 

See also: Web App Tracks Breaking News Using Wikipedia Edits [Link]