Robert Steele: Reflections on China, Open Source, & the Craft of Intelligence UPDATE 1

#OSE Open Source Everything, All Reflections & Story Boards
Robert David Steele
Robert David Steele

The below article is important to those thinking about open source software in Europe:

Why China is the next proving ground for open source software

The article, from TechRepublic, focuses on the Chinese taking the lead in open source software related to big data infrastructure.  “China scale” is a term that should be noted.

My comments:

01: China, Iran, and India are all at various stages of software development, at this point still in competition with one another. If and when they decide to collaborate, Western proprietary software will die overnight.

02 Most have lost sight of the low-key announcement by the BRICS that they intend to create an alternative Internet. Given the insecurity of the Western Internet including SCADA systems foolishly linked to the Internet, and the increased “control” over language and content being imposed by censors within Google, Facebook, Twitter and LinkedIn, what this really means is that the way is open for an UNCENSORED but SECURE Internet built in Europe. The de-Americanization of communications and computing software world-wide should be a priority, along with the avoidance of an alternative Balkanized Internet sponsored by the BRICS.

03 Big Data today is a fraud. We process less than 1% of big data in hand, and that big data in hand is less than 1% of the big data potentially collectable and exploitable. Exascale processing is unlikely in our lifetime unless a distributed uncensored secure Internet is created that provides rights of anonymnity, identity, privacy, and security to individuals, eschewing the mass surveillance culture of the Western governments.

04 We still do not have a desk-top integrated analytics tool-box that enables informaton sharing and sense-making across all boundaries. The CIA's 1989 unclassified publication, Computer-Aided Tools for the Analysis of Science & Technology (CATALYST) remains the best requirements document available.  I provide additional current (2015) comments in my Forword to the book, Cyber-OSINT: Next Generation Information Access. Creating an open source software AND open source hardware analytics tool-kit that is free to every INDIVIDUAL, should be a regional priority.  It should join Open Access, Open Cloud, and Open Spectrum as a regional competitiveness initiative.

05 The fastest  way for Europe to achieve all these goals is to create an Open Source (Technologies) Agency in partnership with China and India and I would go so far as to also suggest Iran, Russia, and Turkey as well as Malaysia and Indonesia. We divide the world at our peril. My memorandum to Vice President Biden is still available online for exploitation by anyone. The Americans refuse to take open source seriously because vendors own the US Congress and the US White House and they will pay to the death of all of us for the right to continue looting public treasuries instead of providing integrated open source solutions helpful to humanity.   There are 9 major open source categories, 27 critical sub-categories — I have listed them at the P2P Foundation Category:Open Source Everything, but there is no government anywhere that a) understands this or b) is addressing open source as a universal ecology. That is the next big leap, in my generally humble opinion.


See Also:

Reflections: Philosophy of Intelligence


Posted by Szabolcs Szekacs on September 23, 2016 at 13:49

Interesting article, even more interesting the comments!

01: the article talks about the advance of OSS in platform software – there a completely agree with you. However, I am not so sure about generalising this phenomen to all software in general, especially those specifically made for the public sector. I don't see how a huge generic shift in china would have a significant impact here in europe for the public sector.
Administrations do not even reuse each others' solutions within one country (i.e. in the same legal, social environment).
While we have great cases of cross-border collaboration in Europe (aa reported by OSOR and Joinup), these are still “good practice” cases and not “normal ones”.

03: about exascale: maybr I am younger or more optimistic than you, but i think exascale will happen within 10 years if not 5. (us department of energy is signing contracts with the aim of developing two such systems by 2023). If you mean exascale computing which is accessible to more than a dozen institutions / companies on earth, that's another question.

Posted by Robert Steele on September 24, 2016 at 7:13

I am delighted to have a comment from an informed practitioner. I reply to your two comments and add a note.

01 I was one of the original CIA “digital innovators” in 1986, they built the Artificial Intelligence Staff in the Office of Information Technology around me (I was an over-educated clandestine case officer – spy – who was sent there to shake them up*). What I learned was that data entry — one time acquisition, infinite access — is the choke point. The proprietary software world has Balkanized the data world — fortunately this is changing now at both the software (open) and access (SciHub/EU 2020) levels. Most people do not realize that the real problem is not at the national level but at the local level. Being able to integrate data from across 18,000 police jurisdictions in the USA, to take one example, has proven impossible for the FBI, so they have given up and completely lost sight of — to take one example —  the systematic murder of medical and environmental and some political activists. I would go a step further and observe that the real challenge is not within government, which operates on roughly 2% of the relevant information, but in sharing across the eight tribes of information (academic, civil society including labor unions and religions, commerce especially small business, government especially local, law enforcement including private security companies, media including bloggers, military including gendarme, and non-government/non-profit from the UN to the local soup kitchen). A useful graphic is here: Graphic: UN and eight tribes of information-intelligence, this applies equally well to the EU and to national governments.  I  totally agree with you on the current dysfunctionality of sharing, but with the coming financial collapse I see a new opportunity. The EU was correct to refuse to create an EU spy agency — as I published in Defence and Intelligence Norway, this was the right decision, wrong question, a missed opportunity. What the EU needs is the Open Source Agency starting at Euro 125 million in year one and building rapidly toward Final Operating Capability of Euro 2-3 billion in year six.

03 On exascale, I think I provided a link to a write-up on why not in the original post but if not, here it is again. The problem we have today is that there is no excess processing capacity. Everything is focused on collection and storage.  We need to be able to process — make sense of — all information in all languages and mediums in near real time and this includes the 80% or more that is not digital at this time. The National Security Agency admits it processes less than 1% of what it collects (but if you are prime minister be assured all your screen views and emails are in that 1%); Google admits it indexes less than 4% of the web (I think it is closer to 2%); Mary Meeker observes that we process less than 1% of Big Data (which is generally 1970's in its collection schema and useless for holistic analytics and true cost economics because no one cares about those aspects of information exploitation) and so on. For exascale, I completely agree that we could have a break-through, it would have to be rooted in a global distributed blockchain Open Cloud combined with Open Spectrum combined with a globally available low to free Open Office that includes the 18 analytic functionalities including pattern analysis and anomaly detection, as called for by the 1989 unclassified CIA document, still not available because industry profits from collection and storage and they know that processing and analysis would lead to the cutting of budgets in half as the 50% average waste becomes quickly visible in a compelling manner that would mobilize violent public opinion if not stopped.

In my view, the EU needs to make Open Source Everything Engineering (OSEE) and the idea of an EU Open Source Agency, an agenda item for the new EU President. A starting point is provided by the document I prepared for my lectures in Oslo and Copenhagen in April 2016, Open Source Everything Engineering (OSEE): A Nordic Manifesto. There is strong Nordic interest in a multinational inter-agency informaton sharing and sense-making centre and network, but I have not found a source of leadership yet, that is what is missing now.

I am easily available to you via skype, and would be quite delighted if we could elevate this discussion to the high table. My personal web page including contact information is

* Since 1988 I have become the leading intelligence refomer in the English language, and the father of the modern Open Source Intelligence (OSINT) movement. My most famous sound bite is “do not send a spy where a schoolboy can go.” My eight books on intelligence reform — two with Forewords from Senators themselves Chairmen of the Senate Select Committee on Intelligence (SSCI) — all suggest that we need to cut the secret services back to 20% of their present dysfunctionality, while applying the savings to open source intelligence and open source engineering for the greater good of all members of the public.

Financial Liberty at Risk-728x90