Okay. You got me. I can’t really tell you everything you need to know about big data. The one thing I discovered last week – as I joined more than 2,500 data junkies from around the world for the O’Reilly Strata conference in rainy Santa Clara California—is that nobody can, not Google, not Intel, not even IBM. All I can guarantee you is that you’ll be hearing a lot more about it.
What is big data? Roughly defined, it refers to massive data sets that can be used to predict or model future events. That can include everything from the online purchase history of millions of Americans (to predict what they’re about to buy) to where people in San Francisco are most likely to jog (according to GPS) to Facebook posts and Twitter trends and 100 year storm records.
Phi Beta Iota: Big data is most important for what it can tell you about true cost and whole system cause and effect, inclusive of political corruption and organizational fraud. These are past and present issues, not future issues. We design the future based on the integrity present today. This is why “open everything” matters.
With that in mind, here’s the three most important things you need to know about big data right now:
Syria: Deputy Oil Minister Abdo Hussameddin announced his resignation and departure from the Ba'ath Party to side with the opposition against President Al-Asad's regime. If confirmed, he would be the highest-ranking official to defect, and the third member of the administration to do so. A video of his declaration was posted on YouTube and repeated around the world.
Comment: Most news outlets reported this man as the highest-level official to defect, which means very little. A lengthy search showed the man was a Baath Party member for a long time, but failed to discover whether the defector was a Christian, Druze, Sunni, Alawite or member of another group. The implications of the defection hinge largely on details not available in the public domain.
Syria celebrated the 49th anniversary of the Syrian coup by Hafez al-Asad on 8 March 1963. Revolution Day is 8 March.
Correction: The place names cited by the Red Crescent official and reported in the 7 March edition of NightWatch are governates, not cities and towns. Syria has 14 governates – often translated as provinces – which administer 61 districts.
It is important to enter an instability problem at the right level, meaning at the level of political organization that provides diagnostic and prognostic results. The international press persists in describing unrest in terms of governates. Entering the instability problem at this level results in distorted narratives and exaggerated reports about the strength of the opposition and the weakness of the government.
Readers are justified in wondering why the government in Damascus has not collapsed. The reason is that the government is not now and has never been threatened by a governate-level insurrection. The fight is in local neighborhoods and most are on the political or geographic periphery of the governates, posing little threat to central authority.
Syria is about the size of North Dakota, according to the CIA World Factbook, with a few differences. Syria has 61 districts which more or less correspond to North Dakota's 53 counties. North Dakota's counties, however, are not organized into governates or provinces.
Syria supports more than 22.5 million people in the same space that North Dakota supports just under 700,000, but with a lot less water. North Dakota has no cities as populous as Syria's Homs which contains over a million people. North Dakota has no sea ports or borders with hostile enemy states.
NightWatch has sought to enter the Syrian instability problem at the district or sub-district level so as to guard against bias and get finer ground truth granularity about just what is happening in Syrian neighborhoods.
For example, a careful survey shows that today the Free Syrian Army and its supporting web sites posted situation reports indicating that this force engaged in six operations in five different governates on 7 March. Several were exchanges of gunfire in which no one was injured and one was erection of a roadblock, in a territory the size of North Dakota.
This data supports leaked information attributed to US intelligence persons that there isn't much of a Free Syrian Army. There is unrest in Syria, but there really isn't much of an insurgency. For the purposes of comparison, in Iraq in 2006, more than 300 firefights occurred daily. In Afghanistan last spring, there were around 50 firefights daily and hundreds of incidents involving makeshift explosives.
Syrian security forces were busy. Opposition sources reported dozens of activities in nine of the 14 governates. A closer look showed that the activities were concentrated in about a dozen of the 61 districts.
Nine governates sounds like a big insurrection. Unrest in 12 districts presents a far more manageable security problem than nine governates supposedly out of control, but in fact not. No governates are out of control and apparently neither are any of the 61 districts.
A still finer focus showed that most of the opposition activities were small, brief street demonstrations (which were not further defined), according to the opposition's own postings. There were no clashes except as noted above; no bombings and no terror attacks on 7 March.
Most of the government operations were local neighborhood sweeps that encountered no resistance. Other reported government actions included over flights of aircraft, some vague armor movements and shelling. The opposition sources that posted the reports were not careful to distinguish whether the operations were by law enforcement and police personnel, paramilitary militias or the Syrian armed forces. Most were attributed to “thugs,” which suggests the paramilitary militias.
Unfortunately the sources also were not specific about which sub-districts or neighborhoods were under stress from government operations. Each of the 61 Syrian districts has multiple sub-districts what are called, nawahi. It is not yet possible to track activity at the nawahi level, but it would show a more fine grained definition of the status of the instability problem in Syria.
Phi Beta Iota: CNN and BBC both appear to be taking direction from US covert operations / media influence staffs. Both appear unintelligent and dishonest. We hold NIGHTWATCH and its editor in the highest regard, consistently superior to the larger organizations that lack both intelligence and integrity. We note with interest that the Syrian Diaspora and the crisis mapping communities are relatively silent on this matter.
Back in 2002 while I was at a major Command, Friedman made a pitch to provide “ground truth” intelligence on contract. While some folks went gaga over his presentation, there were many small indicators that all was not well. Apparently Friedman was asked to leave LSU for reasons undetermined. Offically, according to STRATFOR ” In 1997, a small company that would eventually grow into Stratfor—called Strategic Intelligence LLC—left Baton Rouge and LSU, where its founder George Friedman had been a professor. A 1999 profile in Texas Monthly said the company “couldn't thrive” in Baton Rouge, and that's why Friedman took it to Austin, where it blossomed into a global powerhouse.”
Reasoning heard on the street by colleagues active in San Antonio, he was bankrolled by Mossad and they wanted him in an area of hightech.
I have read his comments about OSS and you specifically; his arrogance is unbelievable. His “analysts” are currently UT-Austin students with a couple of “seasoned veterans”, none of whom have an intelligence
background.
He has been described as a neocon and I think that fits. From where I sit, I think STRATFOR is finished, they are trying very hard to regain their client base….with little success…
As Wyoming plans for federal collapse (where, exactly, they will put the aircraft carrier remains unresolved), it behooves all of us to spend a little time thinking about “what if” the national supply chains for food and fuel implode.
I am not that enthused about the terms “smart city” or even “intelligent city,” but recognize both among the links below. Neither smart nor intelligent equates to agile, adaptive, resilient, or sustainable.
Phi Beta Iota: As anticipated, nothing really original here. As not anticipated, there is nothing at all from any other discipline. The point that intelligence theory and practice today is retarded, and that all of the other disciplines are themselves fragmented into incoherence, appears to have escaped the organizers.
I have begun drafting my portion of the new Handbook of Intelligence Studies (Routledge, 2013), it is a chapter early on entitled “The Craft of Intelligence.” I pick up where Allen Dulles and Sherman Kent left off. My graphic on Intelligence Maturity captures the essence of my thinking at the strategic level, but of course there is more to come, including the desperate need to restore integrity to all that we do.
In 1988 I ghost-wrote for the Commandant of the Marine Corps an article that he enhanced and signed, “Global Intelligence Challenges in the 1990's.” At that time my focus was on the difference between the conventional threat and the emerging unconventional threat.
Now my focus is on the purpose and process of intelligence as decision-support. We must — we will — move from secret intelligence for the few to open intelligence for the many; from expensive centralized largely worthless intelligence to free and low-cost distributed intelligence relevant to every person at every level on every issue; from intelligence as window-dressing for channeling $80 billion a year to banks and corporations, to intelligence as an integral element of every aspect of a Smart Nation.
Today Owl sent me a link to an article, Philip E. Tetlock and Barabara A Mellers, “Intelligent Management of Intelligence Agencies,” American Psychologist, 2011, pp. 1-12. I respect Owl, so I printed it and read it twice.
This article is completely out of touch with reality and the authors have not bothered to familiarize themselves with the literatures pertinent to their endeavor. Out of 89 cited sources 12 are non-intelligence-related prior publications of the lead author, 1 is a prior publication of the second author, and 11 are ostensibly about intelligence but truly marginal selections. So 12% sources on the subject, 13% self-citation, and 75% escoteric psycho-babble irrelevant to the actual challenge. As an intelligence professional, I am offended that two ostensibly erudite individuals would dare to publish this trype without even a semblance of understanding of the subject under discussion.
BCISS Conference Materials: Understanding and Improving Intelligence Analysis
Current Intelligence Reporting and the H5N1 Virus (Vogel)
Seeing Is Not Believing (Lawhead)
