We test not simply because we enjoy it; we test because it informs us. Tests grant us insight into how the Google algorithm works and on what we should, and more importantly should not, be focusing our energies. Tests also allow us to make the case for what SEO best practices to be implemented into the early stages of a project. Without concrete evidence, why should project managers grant us SEOs a seat at the table? Their job is to get a project accomplished as quickly and efficiently as possible. The more stakeholders involved, the more complicated and more slowly, a project takes is completed.
Just recently, I was speaking with my UX and page editors explaining to them the importance of a good hierarchical structure using header tags (i.e. H1, H2, H3, etc.). One of the members pointed to a tweet John Mueller had posted last year that implies that placement of headers tags on a page is not a ranking factor. It was the coders’ belief that Google can “figure it out” and SEO doesn’t need to be involved in page design.
Source:https://twitter.com/JohnMu/status/1023571134699188225
After reading the tweet myself,the idea that Google does not need any guidance is not peticularly what I took away; but I can see how someone could be left with that impression. This takes us back to my first statement. If we want to be included in the project planning phases of site or page development, we need to be able to prove the value of SEO with data. Lucky for us, we can run tests to show how the search engine will react to different kinds of pages structures.
Proving that structure matters is the basis of this month’s tests. As always, there are dozens of possible ways we could tackle this subject. I ultimately ended up on creating 7 variations of this test.
Test #1 – Testing if the presence of an un-optimized H1 tag is a ranking factor. I see this as establishing a base argument for using H1 tags.
Tests #2 & #3 – Test if the position of the un-optimized H1 tag on a page is a factor when no other header tags exist. Does Google care about H1 tags if no other tags are being used?
Tests #4 & #5 – Test if the position of the un-optimized H1 tag on a page is a factor when H2 header tags are also present. Does Google care about proper usage of H1 tags if other tags are in place?
Tests #6 & #7 – Test if position of the un-optimized H1 tag is a factor when a complete page hierarchy is in place – specifically H2s and H3s are on the page. Does Google expect you to order them properly?
Test #1
I setup 7 identical pages with a shared fake keyword. One for which there are no results when you search Google. Each page had4 paragraphs (using <p> tags) and above each paragraph, plus one at the end, was a single word wrapped in a <p> tag itself. Those single word <p> tags were my placeholders for future header tags. I placed random English in all the paragraphs and published the 7 identical pages. After the rankings for the fake keyword settled, I took the middle ranking page (the page ranking #4 out of 7) and swapped the initial header tag placeholder from <p>to<H1>. Then I waited. To my surprise, the page never really jumped or moved significantly. Could it be? Does Google not care about H1 tags as much as I thought?
Tests #2 & #3
After the initial test result, I setup two more setsof 7 pages,each set with its own fake keyword. Using random English words, I created the two sets of pages so that each set contained identical copy. The pages contained header tag placeholders before each paragraph plus one at the end. I published the pages, and waited for them to be indexed and for the rankings to settle. Once the rankings settled, I adjusted the middle ranking page (4th position out of 7) within each set. In test #2, I swapped the central header tag placeholder from <p> to <H1>. While in test #3, I replaced the last header tag placeholder from <p> to <H1>.
After waiting a week or so the results were in and very interesting as well. In test set #2, I added an H1 tag to the middle of the page. The same amount of text was both above and below the H1 tag. This page’s ranking did not move. In test set #3, I added an H1 tag to the very bottom of the page. All page copy was above the H1, none was below it. This page dropped to the bottom of the rankings decisively and immediately, it never fluctuated.
I found the results of the 3 tests a little perplexing. The structure of the page did not seem to matter that much but there is a way to “do it wrong”. Google’s algorithm seemed forgiving of pages that did not have an H1 at the very beginning of the content but clearly did not like it at the very end. My initial thought was maybe this was because we only had an H1; there were no other tags (like H2 or H3). So I decided to re-run a version of the last two tests (#2 & #3) but instead of placeholders using <p>, there would be <h2>tags instead.
Tests #4 & #5
I setup two more sets if 7 pages each; like the previous tests each had a unique fake keyword. Using random English words, I created the two sets of pages so that each set contained identical copy. Also, just as in the other tests the pages contained header tag placeholders before each paragraph plus one at the end. Except this time, I wrapped the placeholders in a <h2> tag. I published the pages, and waited for them to be indexed and the rankings to settle. Once the rankings settled, I adjusted the middle ranking page within each set. In test #4, I swapped the central header tag placeholder from <h2> to <H1>. While in test #5, I replaced the last header tag placeholder from <h2> to <H1>.
After waiting a week or so the results were in. In test set #4, I swapped the middle H2 for aH1. So now, we had two H2s and two paragraphs above and below the H1. The page ranking wiggled a little but never jumped to the top or bottom of the rankings. I suspect this was SERP “noise”. In test set #5, I added an H1 tag to the very bottom of the page. Four H2 tags and all page copy was above the H1, nothing was below it. This page dropped to the bottom of the rankings but often moved up one or two to drop again. Like test #3, I am going to call this a negative move and chalk up the minor movement to SERP “noise”as I saw in test #4.
Now is a good time to mention the occasional challenges when testing. I started these tests in March, when Google began running some kind of algorithm update. This caused me all kinds of trouble. Initially the pages were not indexed. Usually I can get them indexed within a couple days but this time it took a couple weeks at least. Then, it took another couple weeks for the rankings to settle. Interestingly enough though after I made the adjustments there was little fluctuation, except for the last set. I suspect this was due more to timing than any other factor. One of the many challenges you can face while testing.
Tests #6 & #7
These results did not really satisfy me either so I decided to develop two more last tests. This time the pages will have a full header tag hierarchy implemented. I setup two sets if 7 pages each, just like the other tests using random English words and a fake keyword. Unlike the previous tests, the pages contained a standard header tag hierarchy – H1 at the top, three paragraphs preceded by H2 tags and a H3 tag at the very bottom of the page.
I published the page sets and then waited for them to be indexed and the rankings to settle. Once the rankings settled, I adjusted the 4th ranking page within each set. In test #6 I swapped the H1 tag and the second H2 tag. The page now has two H2 tags, an H1, and lastly one more H2 tag then an H3 at the end of the page. The hierarchy is out of order from best practices, which should hurt the page’s ranking. In test #7 I swapped the H1 with the H3 tag at the end – this page’s hierarchy is completely out of order.
After a couple weeks of fluctuations, the rankings finally solidified. As expected, both of the variations dropped to the bottom of the rankings. Google’s algorithm clearly did not like the page structure being “out of order”.
Unlike previous tests, I found this one to be a little less decisive. Partly because I experienced so many odd fluctuations in Google my confidence in the results are reduced. Part of being a good tester is to constantly question your methods and look for ways to improve. Going forward I will start running an inverse test for every set – swapping the control and variations in each. For example, in test #1 I tested adding an H1, the inverse test would be to have H1 tags in all the pages and remove it from one page. With this method, if adding the H1 to test #1a moved the page up (even a little) and if removing the H1 from a page in test #1b moved the page down the rankings, I would have a much higher confidence in the results.
So can we take anything away from these tests? I would say yes, it did yield some interesting clues, just not the ones I expected. The mere presence of an H1 tags does not seem to help a page rank better. Placing content above your H1, does not seem to negatively affect your rankings either. With that said, you can “do it wrong” if you place the H1 tag at the very bottom of the page; that clearly was a negative factor. In addition, if the page has a fully developed hierarchy the algorithm seems sensitive to the order as well. Therefore, John Mueller’s tweet was somewhat accurate but also did not tell the whole story. This is how myths get started and non-SEOs can easily get confused.
The belief that Google can “figure it out” is true ONLY IF your page does not have full structure. Too often, SEOs present themselves in a mystical fashion, this causes some to be suspicious and exclude us from important planning sessions. We can use a robust testing process to illustrate these nuances to designers and developers and earn our seat at the planning table. We can use our knowledge to help inform the process but not hijack it. While there does seem to be a right and wrong way to structure the page Google is also flexible. Like a good lawyer, we can use our knowledge to keep designers/coder out of trouble without needing to place heavy directives on the team and slow down the project.