Websites have always had ranking problems – an inability to break into the top 10, then top 3 for search queries that would grow their business. Increasingly however, the conversation around this has changed. More and more, people are discussing indexation, not ranking.

Table of Contents

Is Content Still King?

To be sure, Google has never indexed every single page made available to them. And that’s whether the website owner wanted them indexed or not. Many things have changed in the past couple of years making this problem more palpable.

For one, the push to create content is larger than ever. While the expression “content is king” is hardly new, SEO’s around the globe have been extolling Expertise, Authority and Trust for a few years now, and in part, you can’t create those 3 things without content.

The Push for More Web Content

And people have taken that to mean: Lots of content.

I think we’ll find in the end this becomes a Quality over Quantity discussion.

We need to redefine what a kingmaker is.

As I mentioned in “Why isn’t Google Indexing my Content” in early May 2022, the web is growing at an incomprehensible rate. By some estimates, as many as 500,000 websites are created every day. 

Not pages… Websites.

And then we have a few other factors.

Why is the Web Growing so Fast?

A few reasons.

Social Media

First is social media. If you consider that every YouTube video, every Tweet, every Facebook rant has a URL… That’s a page. And they get crawled, since you see them in search.

The “Ease” of Doing E-Commerce

Second is that e-commerce is now so unbelievably accessible. Tools like Shopify and BigCommerce make it super simple to set up shop, and there’s apps to basically let you import products to dropship at the click of a button (and the swipe of a credit card).

Your SEO, Inhouse or Agency

Third is us- Your friendly SEO partners. We tell you over and over “you need more content”.  And that’s sometimes true. But the shorter path to success we should be guiding you to is:

“You need content that is unique, relevant, and important. You need to publish it on a technically sound platform that allows search engines to ingest it, understand it, and feel like ranking.”

Programmatic SEO

Next would be the ability of Content Management Systems to leverage data to do programmatic SEO. You can literally take a Google Sheet and turn it into a 50,000 page website in a matter of hours. Not a good one, but a website none the less.

Artificial Intelligence

Last, we now have the ability to generate content that is pretty good with artificial intelligence. If no one has to write it, there’s no “cost” to creating it and publishing it. That will change quickly if Google decides to penalize websites it detects are using AI to generate content for the sole purpose of improving their rankings.

Is it Good that the Web is Growing So Quickly?

It would be if everyone was bringing their A Game.

The challenge for search engines is of course the volume they have to keep up with. On top of that, they need to make value judgments in near real time as to the worth of one piece of content over another. That becomes mathematically complex as the numbers grow.

It’s Not All Unique. It’s Not All Valuable

If all content were truly unique and valuable, and offered a new solution to a problem, or better information, then this might even be more of a challenge.

The fact is, so much of the content we generate these days is simply rehashing topics, or using canned manufacturer product descriptions.

The Signal to Noise Ratio is Worse than Ever

In that regard, the signal to noise ratio is getting worse. One of the end results is that Google seems to be using a different decision criteria than it used to.

It’s no longer “Does this content deserve to be ranked?”.  It’s now “Do we even have the desire to spend resources crawling this?”

In that regard, we as website owners are doing ourselves a disservice. There’s too many ways we can accidentally make this decision easy for them.

Signal to Noise Ratio in Web Content is Getting Worse

How the Conversation is Shifting

Even in our pretty tight knit online SEO community, I’ve seen more discussion of this. Much like how paid search agencies have had to adjust to Google’s dramatic recent changes in advertising, SEO’s are starting to look deeper at new types of crawl reports rather than just ranking reports.

So there’s work being done.

Correlation Studies on the Indexation Topic

I ran in to a great correlation study by Onely this past week. Correlation studies are problematic for sure, as the takeaways are not 100% scientific. In the case of indexation, however, we don’t have the body of work like we do for ranking studies.

Check out the link to their study. It’s a good read, and at the very least thought provoking. And they’re quick to point out where an assumption is based on a correlation, which is good.

Some quick takeaways:

  • Even for very popular e-commerce sites, nearly 22% of their content in their XML sitemap is unindexed
  • The amount of unique content on a category page seems to play a strong role in getting it indexed
  • The number of pictures for a product correlates to a higher indexation rate

What to do Next

If you’re here reading this, it’s possible that you are experiencing Google indexing issues yourself. The first thing to do is read our post on “why isn’t Google Indexing my content?”

If the problem is a little deeper than that, I’d highly recommend you engage us for a deeper analysis of your website through our e-commerce SEO Audit and Roadmap.  We’ll get under the hood of you and your competitors and give you a concrete path forward!

Kevin Webster

I've been doing digital marketing since 2003. I've worked with clients across dozens of verticals, helping them achieve results in SEO, Paid Search, and analytics.

Leave a Reply