I recently had the distinct honor of being on the opening plenary of the 2012 Skoll World Forum in Oxford. The panel, “Innovation in Times of Flux: Opportunities on the Heels of Crisis” was moderated by Judith Rodin, CEO of the Rockefeller Foundation. I’ve spent the past six years creating linkages between the humanitarian space and technology community, so the conversations we began during the panel prompted me to think more deeply about innovation in the humanitarian space. Clearly, humanitarian crises have catalyzed a number of important innovations in recent years. At the same time, however, these crises extend the cracks that ultimately reveal the inadequacies of existing humanita-rian organizations, particularly those resistant to change; and “any organization that is not changing is a battle-field monument” (While 1992).
These cracks, or gaps, are increasingly filled by disaster-affected communities themselves thanks in part to the rapid commercialization of communication technology. Question is: will the multi-billion dollar humanitarian industry change rapidly enough to avoid being left in the dustbin of history?
Crises often reveal that “existing routines are inadequate or even counter-productive [since] response will necessarily operate beyond the boundary of planned and resourced capabilities” (Leonard and Howitt 2007). More formally, “the ‘symmetry-breaking’ effects of disasters undermine linearly designed and centralized administrative activities” (Corbacioglu 2006). This may explain why “increasing attention is now paid to the capacity of disaster-affected communities to ‘bounce back’ or to recover with little or no external assistance following a disaster” (Manyena 2006).
Okay. You got me. I can’t really tell you everything you need to know about big data. The one thing I discovered last week – as I joined more than 2,500 data junkies from around the world for the O’Reilly Strata conference in rainy Santa Clara California—is that nobody can, not Google, not Intel, not even IBM. All I can guarantee you is that you’ll be hearing a lot more about it.
What is big data? Roughly defined, it refers to massive data sets that can be used to predict or model future events. That can include everything from the online purchase history of millions of Americans (to predict what they’re about to buy) to where people in San Francisco are most likely to jog (according to GPS) to Facebook posts and Twitter trends and 100 year storm records.
Phi Beta Iota: Big data is most important for what it can tell you about true cost and whole system cause and effect, inclusive of political corruption and organizational fraud. These are past and present issues, not future issues. We design the future based on the integrity present today. This is why “open everything” matters.
With that in mind, here’s the three most important things you need to know about big data right now:
A new program can find and compare relationships in complicated data without having to be asked specific queries
Are there subtle patterns lurking in data that can foretell of a coming financial-system crash? What can explain the variations in sports-star salaries? How about the complex relationship between genes and certain diseases? Scientists in various fields have been searching for better ways to analyze large piles of data for such patterns, but the difficulty has always been that they need to know what they’re looking for in order to find. A new software program, described in the latest issue of Science, is designed to find the patterns in data that scientists don’t know to look for.
David Reshef, one of the scientists behind MINE, as the program is called, explains, “Standard methods will see one pattern as signal and others as noise. There can potentially be a variety of different types of relationships in a given data set. What’s exciting about our method is that it looks for any type of clear structure within the data, attempting to find all of them. … This ability to search for patterns in an equitable way offers tremendous exploratory potential in terms of searching for patterns without having to know ahead of time what to search for.” MINE compares different possible relationships (including linear, exponential, and periodic) and returns those that are strongest.
How does public transportation affect education? What impact does population density have on public health? Is there a connection between CO2 levels and obesity?
Officials in the City of Portland, Ore., have collaborated with IBM to find answers to those and other questions, developing an interactive model that connects the relationships between the city’s core systems that handle the economy, housing, education, public safety, transportation and health care.
Is there a new anti-intellectualism? I mean one that is advocated by Internet geeks and some of the digerati. I think so: more and more mavens of the Internet are coming out firmly against academic knowledge in all its forms. This might sound outrageous to say, but it is sadly true.
Robert David STEELE Vivas: Digerati are not geeks. They are adept at social media, a process, rather than the substance of any discipline. Their scorn for the mandarins of knowledge would not be possible if academia had not lost its soul, sanctioned massive intellectual corruption, and fragmented itself to the point of irrelevance. The serious educational literature (not something the digerati read) is clear: inspiration and innovation
emerge faster, better, and cheaper from minds that are prepared, to include a foundation of memorization and a deep familiarity with the thinking of those who have come before. The digerati point of view half-right and is embodied in Smart Mobs, Wisdom of the Crowd, Everything is Miscellaneous, and Maria Popova’s latest thought, that “information curation is the new authorship.” The digerati approach splits the roles of originator of an idea and connector of an idea down, and assumes that “the collective” can replicate and even surpass the individual human brain, without recognizing that the whole is only as good as the sum of the part foundation plus whatever the collective adds. My own finding re Wikipedia is that the mob destroys intellectuals. My own efforts to enhance the Open Source Intelligence page there were destroyed by idiots that “assumed” that because I pointed to oss.net so much (to many of the 800 people whose work is there including the 144 that received Golden Candle Awards) I was “self-promoting.” The digerati are fragile and very shallow, and by Larry Sanger’s very interesting account, a new form of neo-Luddite. The academy is corrupt and fragmented–we are in an era where all forms of organization have lost their soul and whatever semblance of philosophical context they may once have possessed. We are suffering from the Paradigms of Failure that I discussed in the pre-amble to ELECTION 2008: Lipstick on the Pig (EIN, 2008). There is only one option leading to stabilization & reconstruction: INTEGRITY. The digerati aren’t–as a general rule–very appreciative of holistic thinking or in-depth expertise–they are a spoiled generation badly in need of some personal suffering and exposure to global reality–IMHO.
“The nature of cyberspace is borderless and anonymous,” R. Chandrasekhara, secretary of India’s telecommunications department, told a cyber security conference in London last week organised by a U.S.-based think tank, the EastWest Institute. “Governments, countries and law — all are linked to territory. There is a fundamental contradiction.”
Phi Beta Iota: The national secret intelligence communities mean well, but they are cognitively and culturally incapacitated in relation to both the global threats and the global infomation sharing and sense-making possibilities. It may just be that the solution has to come from a private sector service of common concern that can provide the integrity now lacking in governments and most corporation. Scary thought. M4IS2 is inevitable….delay is costing trillions.
Phi Beta Iota: All of the posters (use link) are expandable and printable. Below are just a few examples. Visit the 2011 GIS Poster Expo Gallery for all the others. These are generally GIS 2.0, and do not yet focus on “deltas” between collected data sets (e.g. Chinese investment and corruption), nor do they question standing assumptions (e.g. that wind farms are the answer vice individual windmills).
Here are a few of the BIG lies used to support the status quo. What we need, rather urgently, is a counter-narrative
LIE 1. The earth is an open system with infinite supplies and sinks;
POSSIBLE TRUTH: Earth is a closed system, changes that used to take 10,000 years now take three, humanity is “peaking” the entire system.
LIE 2. Everything must be monetized;
POSSIBLE TRUTH: Money is an exchange unit and an information unit; in the absence of holistic analytics and “true cost” transparency, mony is actually a toxic means of concentrating wealth and depriving communities of their own resources (e.g. land).
LIE 3. The extreme unregulated free market is the only option for a modern economy;
POSSIBLE TRUTH: Information asymmetries and “rule by secrecy” have been clearly documented–the free market is neither free nor fair. A modern economy needs to be transparent, resilient, and hence rooted in the local.
Before Gutenberg, a book cost about as much as a small house. As a result, only kings and bishops could afford to own a book of their own.
This naturally led to the creation of shared books, of libraries where scholars (everyone else was too busy not starving) could come to read books that they didn’t have to own. The library as warehouse for books worth sharing.
Only after that did we invent the librarian.
The librarian isn’t a clerk who happens to work at a library. A librarian is a data hound, a guide, a sherpa and a teacher. The librarian is the interface between reams of data and the untrained but motivated user.
After Gutenberg, books got a lot cheaper. More individuals built their own collections. At the same time, though, the number of titles exploded, and the demand for libraries did as well. We definitely needed a warehouse to store all this bounty, and more than ever we needed a librarian to help us find what we needed. The library is a house for the librarian.
Humanity and technology continue to co-evolve at an ever increasing pace, leaving traditional institutions (and mindsets) calcified and out of date. A new paradigm is emerging, where everything is increasingly connected and the nature of collaboration, business and work are all being reshaped. In turn, our ideas about society, culture, geographic boundaries and governance are being forced to adapt to a new reality.
While some fear the loss of control associated with these shifts, others are exhilarated by the new forms of connectivity and commerce that they imply. Transactions and interactions are growing faster and more frictionless, giving birth to what I call a “superfluid” economy.
Business will not return to usual. So let’s discuss 4 key concepts to help us better understand the shifts that are underway:
1. Quantifying and mapping everything
2. Everyone has access to the internet
3. Self-organizing expands
4. Peer-to-peer exchange changes the future of money