I have known Stephen E. Arnold since the early 1990's when he quickly became the only speaker to be demanded by my international audience every year. I have relied on him over the years for truthful open source intelligence and deep insights not available from other commercial sources. Although he is now retired, I believe he will be instrumental in guiding millions of dollars in investments intended to create a national truth channel and social ecology that cannot be censored.
Here is my favorite bit up front:
“There is not enough money available to start over at Google. After two decades of fixing, tweaking, and enhancing, Google search is sort of chugging along. I think it is complex and swathed like a digital mummy in layers of code.”
Interview below the fold.
What is the basis for your deep knowledge of the Google enterprise in all its forms?
I was paid to write a series of reports about Google for my commercial clients beginning in 2002. I created from the non embargoed content The Google Legacy (which documented that Google was going to leave behind the cloud, centralized approach to content services), Google The Calculating Predator (which explained how Google was engineering a new type of lock in based on user data and free services), and Google The Digital Gutenberg (which explained that Google was one of the largest publishers in the world because every bibliographic entry, every result page, every link to a video is a published content object with ads and data capturing capabilities. I quit writing about Google after I did a series of columns, now on the Information Today web site about Google. I just got bored and shifted to what I call Cyber OSINT.
Can Google rig search results?
Yes and no. Plus it depends on whom you ask. And what’s “rig” mean?
How does this search intervention work?
Scripts, stored queries, canned pages, and management console tools. Keep in mind that Google’s search plumbing is very complicated and quite old. Also, over the years software widgets and wrappers have been coded to deal with special situations like the death of a popular music personality or a major event like a volcanic eruption. Add a word to the forbidden term list or tweak a threshold setting. Bingo. Search results can change.
Can result shaping be automated?
Sure, rules make it possible for Google to show search results for me “Stephen E Arnold” with a photo and information about my work along with search results shaped to emphasize what people find useful in my work. Google assigns me the status of a B list celebrity and my result pages reflect certain content.
Do you know of any Google search result manipulations?
I have heard about problems with Foundem (a UK site), a site offering French income tax forms, and a US site down checked for running non Google ads. Can these allegations be proved? Not unless Googlers talk about search results.
What does SEO (search engine optimization) do?
The idea is that words, phrases, and overall content can make a result appear at the top of a search result list. This type of shaping was and is mostly hit and miss. It is much easier to buy clicks via Fiverr for YouTube views, pay for a news release to be pushed into the news channels, and buy AdWords which point to a Web site.
What is Google PageRank?
The Google PageRank system is a popularity contest with some razzle dazzle added. Few people know or care that PageRank seems to have been influenced by IBM Almaden researcher Jon Kleinberg’s CLEVER system and method. The idea is that sites which get clicks point to other sites. The more links and clicks tells the system that a particular page, object or site is important and should, therefore, be listed at the top of a results list.
And SEO?
SEO tries to deliver a boost, but Google wants people to buy AdWords. The results can be fiddled a bit, but the best way to be visible is to buy AdWords.
Let me be direct as is my custom. Can Google intervene so a search result or site is down checked or buried?
Sure, that’s how terrorism, hate speech, and other content is “disappeared.” There is a list of words that will force down checking at this list of bad words banned by Google.. Also ignore the list of “quality” items in The Google Legacy and lose ground in search results rankings.
Do you know of an instance where Google pushed a site off its service.
Yes, after I wrote The Google Legacy in 2003, a consultant asked me to look at a web site which just disappeared from the Google index. I am not permitted to mention the consultant or the Web site. The site disappeared because the company made many changes to its Web site in a short time. In The Google Legacy I listed about 90 factors which Google allegedly uses to grade a site and rank it. The problem was that the disappeared site violated the guidelines I compiled. Now I think Google publishes its own list of dos and don’ts.
What was the fix for this company’s delisting?
The company had to get a new domain name and start over. As far as my team could determine, this was an automatic action by Google. The company was behind the eight ball.
Who controls search results?
Google usually says, “Our smart software.” The answer based on my research is that some search engineers can adjust thresholds, push popular pages to a cache to deal with a surge of interest in a hit song, or deal with a special situation like the Hawaii volcanic eruption.
Where is the control for search?
Work at Google is a bit ad hoc and sometimes appears to be chaotic. Google works on search in different offices across every time zone 24×7. In many Google offices are search engineers. These individuals can make changes to deal with routine and exceptional issues. For example, the artist known as Prince dies and the Prince queries flood in. That may require manual intervention. Content has to be assembled, maybe manually cached, and possibly organized into a mini-report like the search results for my name.
What about the influence of AdWords?
Revenue is a big deal. Costs are very difficult for Google to control. Therefore, big advertisers have figured out that messages have to be supported by landing pages and other online activities. From my point of view, a big ad buy has an impact on results lists and what users see in results lists. Also, ads may have an impact on what appears in a result list or a YouTube list of suggested videos.
How does Google keep its search system state of the art?
There is not enough money available to start over at Google. After two decades of fixing, tweaking, and enhancing, Google search is sort of chugging along. I think it is complex and swathed like a digital mummy in layers of code. There are some interesting dependences which can surprise on occasion; that is, run a query for specific content, and it is not in the index or a site is a loser and not indexed very often or at all. The US government Railway Retirement Board site comes to mind.
Can an engineer fiddle with search results?
It is possible for an individual with appropriate access to use administrative controls to “tweak” a setting or add a word to the banned word library. Little changes can have interesting and unexpected consequences. Logs keep track of changes, but Google is not managed in the way an old line company like IBM or Oracle is. Google can be quite relaxed and informal in some administrative tasks.
Is there a way to audit Google search results?
Sure, but it takes time, money, and expert searchers, not MBAs who say, “I am a great online searcher.” We completed a project for an automotive Web site, and it was clear that the site was effectively blocked when keywords were used in a query without quotes. Insert quotes around the exact phrase one wanted to match, and the Web page ranked number one in a Google results list. We could not figure out if this was a bug, a feature, or an intentional action triggered when the site owner refused to increase his AdWords spending.
Have you shaped results when you were working on the US government search system?
Yes, Vice President Cheney informed me that his wanted the vice president Web site to appear along with search results for President Bush White House Web site. We did this with a script and a stored query. Search for White House and get hits from Vice President Cheney’s Web site. (I used to work at Halliburton, so I got the request and made it happen within an hour.)
Were you using Google for that US government service?
No, Google did not win the contract because at the time, the company refused to support truncation. So we were using the FAST Search & Transfer system which made it possible to force a query to return specific results and display them on a results page. I assume that other Web search systems support this type of results shaping.
Did you work for Google?
I am not permitted to answer that question.
Are you working now?
No, I retired in 2013. Once in a while I will do a small project.
How can one keep up with your research?
Search for my name on Google or search for DarkCyber, my weekly video DarkCyber. I highlight three to five stories which are important in my opinion and representative of what I put in my books and reports over the years. You can also read my blog at www.arnoldit.com/wordpress. Or just search Bing, Google, Qwant, or Yandex for Arnold Beyond Search. No quotes required by the way.
Thank you.
ROBERT STEELE: Stephen E. Arnold is a regular contributing editor. We value his insights in part because despite spending trillions of dollars on information technology, the US Government is still incapable of accessing 80% of the relevant information in the world; processing more than 1% of what it does access; or making sense of any of the information in a reliable holistic manner that integrates true cost economics, a term the USG does not recognize.
See Especially:
2015 Robert Steele – Foreword to Stephen E. Arnold’s CyberOSINT: Next Generation Information Access
Robert Steele: Core Works UPDATED with 21st Century University
See Also:
Stephen E. Arnold @ Phi Beta Iota
