The last couple of years have been an interesting time for search and politics. Not separately mind you, I am referring to the politics of search. On several occasions, Google has been accused of political bias (most recently and energetically by President Trump) and Google has even spoken before Congress on the subject. I realize any political topic will get touchy but I could not think of a more fun subject to try to test empirically.
It’s not much of a secret that Eric Schmidt, former Executive Chairman of Google (and then Alphabet) both endorsed former President Obama and had a close relationship with his White House and administration. As well, it is known that Mr Schmidt was an avid supporter of Hillary Clinton’s campaign. Of course, there is nothing inherently illegal or unethical about this, nor should there be, people are allowed to have and support their political beliefs. I can of course see why folks on the other side of the political aisle may be concerned. Add to this the fact that employees of Google have been discoveredsuggesting they should adjust search results to favor specific websites in response to President Donald Trump’s immigration travel ban and you have a makings of a great conspiracy theory.
While all of this political intrigue may be fun to discuss and argue about it,the discussion really boils down to one thing: is there a political bias coded into Google’s algorithm. The more I thought about it the more I realized that this could become an enormous undertaking. While I initially mapped out over 25 separate tests I could run, I determined that I needed to focus and test individual variables. I decided to first test if the algorithm has a general bias against President Trump and thenif it prefers his opponents over him.
In test #1, I wanted to see if Google would promote or punish a webpage based on it mentioning a specific person. I first wanted to determine if the algorithm contains a political bias or if it treats everyone equally. To run this test I created 7 pages with exactly the same content – titles and meta descriptions need to be unique or else Google will not index the page. I then came up with a unique fake keyword that displays zero results on Google when searched. Lastly, onall the pages, I needed a control name so I created a fake president name “John Q Smith” and placed it in the Title, Meta Description and 4 times in the body copy. One more note, I decided to mix up usage of the name. Instead of just typing “President John Q Smith”, I used variations such as “President Smith”, “Mr Smith” and “John Smith”.
After the pages were launched, indexed and ranking for the fake keyword I noted which page was ranking in the #4 position out of the 7 pages for the fake keyword. After about a week, the rankings usually settle down and a specific page typically ranks consistently in the 4th position. Once this occurred I edited the page to change all the “John Q Smith” references to “Donald J Trump” – and then I waited. If Google is biased against the President them the page should drop in the rankings. Within several days, the page went to the first position for the fake keyword.
Did this mean Google is biased towards President Trump? I found this equally unlikely so I created 2 more variations of this test to confirm. For each of these variations I followed the same procedure. Each set had the same content except for the Title & Meta Description. Each set also had its own fake keyword and I used “John Q Smith” as the control name placeholder. For the first two sets, I decided to test against the well-known Republican personalities Rush Limbaugh and Sean Hannity. Once all the pages were indexed, I waited for the 4th position ranking to settle. After it settled, I updated those pages by adding the new variable names in each set – this took about a week.Both the Rush Limbaugh and Sean Hannity variable pages moved to #1.
Did I just uncover a right-wing conspiracy buried within Google’s algorithm? While I found this even more unlikely than a vast left-wing conspiracy, I decided to create another 2 test variations. I set them up just like the last set, but this time I used the names Barack Obama and Hillary Clinton as my variables. Just like before, after indexing, updating and waiting both the updated variable pages jumped to position #1 (from the 4th position) for their respective fake keywords.
The interesting thing about testing is that I often learn unexpected things about the search engine. In this case, Google is favoring my updated variable pages. To me this indicates that Google favors “freshness”, “uniqueness” or both. I agree that these pages are hardly fresh or unique, but to a computer; they are more fresh and more unique than the control pages. I have seen this in past tests and actually expected to see this result, but decided to run them anyways to establish a baseline.
So test set #1 did not expose a bias against Donald Trump, nor any of my test subjects for that matter. Therefore, from a larger context Google is not automatically demoting content about our President. The next argument could be that Google may be biased against, or biased in favor of, specific people. So for example, it would rather rank former president Barack Obama over current president Donald Trump. I had my next test.
I setup test #2 very similar to the previous test set. Seven identical pages, except for Titles & Meta Descriptions and a fake keyword on each. This time instead of a fake placeholder control name, I used Barack Obama as my control. After the pages were indexed and the rankings settled, I took the page ranking in the 4th position and replaced Mr Obama’s for Mr Trump’s name. After about a week, the page with Donald Trump’s name ranked #1. I then performed the exact same test, but in reverse, Donald Trump was the control and Barack Obama’s was placed into the page ranking in the 4th position. Again, the updated variable page, with Mr Obama’s name, jumped the 1stposition for the fake keyword.
To be extra clear I decided to run two more tests, I called #2c and #2d, to help keep all these tests straight. This time I was doing the exact same tests as just described but instead of using former President Barack Obama’s name, first as the control and then as the variable, I used President Hillary Clinton. Once again, the page with the variable name ultimately ranked 1st for the fake keyword. Therefore, the Donald Trump variable beat the control pages (i.e. the Hillary pages) and the Hillary Clinton variable pages beat the Donald Trump control pages. I was definitely seeing a pattern here.
After running 9 different tests (5 from test #1 and 4 from test #2) the page that was most fresh and most unique won, regardless of whose name I used. What, if anything, have we learned?
First, the results seem to confirm Google’s never-ending counsel that we build the best content possible. The base algorithm appears to be looking for the most fresh and/or unique content available. I spent the past 6 weeks trying to determine if Google is biased and I learned that the adage “Content is King” seems to be true. That is what I like about testing, sometimes I learn something profound and sometimes you just reinforce the basics.
Secondly, I would suggest that the results demonstrate that Google’s algorithm is likely not overtly biased against any person. While I will agree many employees of Google clearly have a political preference; so far I am happy to say that I see no indication of a political bias programed into the algorithm. It could be biased in a more subtle way, which is why I am running further tests.
I am currently running tests on sentiment. The control pages contain either negative or positive sentiment and the variable page is the other. I will test if a page with a positive sentiment towards President Trump, will it hurt or help it in the rankings. Regardless of the results, I will run it against either Obama or Clinton (or both) as well. I suspect the results will be the same as the tests I already ran but you never know.;-)