Stephen E. Arnold: Search Big Data Flim-Flam – And One Open Source Search of Compressed Files with SQL (RainStor)

Stephen E. Arnold
Stephen E. Arnold

Search and Big Data: Been There, Done That

Is the use of search to find information in large collections of content revolutionary? Er, no. What about using search to locate an Internet Protocol address in a repository of monitored email traffic? Er, no.

With the chatter on LinkedIn and the vacuous news releases from some floundering search companies, one would think that gathering up content and running a query was the equivalent of my ancestor stealing and ember and saying, “Look, I invented fire.”


Beyond the rather influential if specious IBM white paper published in 2010 (link is at, a large number of companies continue to position some old as new again.

One interesting twist on the “search is better than SQL” is the useful solution brief from RainStor. In some circles, RainStor has a low profile. In others, the company has caught the attention of some recognized “names” in the Big Data world; for example, Cloudera and Dell. So think Hadoop friendly.

RainStor focuses on cost effective solutions for gathering, archiving, and querying content. Like the old CrossZ technology, RainStor queries the compressed files. There are benefits from this approach. Unlike CrossZ, no proprietary routines have to be run to extract a data cube. The person looking for information can use standard query syntax using SQL, MapReduce, or off the shelf business intelligence tools.

If you are confused by peas-in-a-pod desperate for a cannery with cash, you will want to check out RainStor. The company’s Web site is I would have like RainStor to publish the numbers of their patents that were granted by the USPTO in 2013. The general description here reminded me of several other firms’ systems and methods.

Stephen E Arnold, April 12, 2014

Opt in for free daily update from this free blog. Separately The Steele Report ($11/mo) offers weekly text report and live webinar exclusive to paid subscribers, who can also ask questions of Robert. Or donate to ask questions directly of Robert.