The first part of understanding your site’s indexing is to have a baseline. This typically will be your XML site map, but what happens when you see something you don’t expect. How few are too few pages in the search engine? We won’t cover how to create site maps or how to get them index in this post, but we will talk about normal indexation as it relates to Google, Yahoo, and Bing.

A site free of technical issues, duplicate content, or thin content will be an ideal site. This site will have few problems being indexed and should have almost all of its pages indexed in Google. Yahoo and Bing have their own process for indexing and typically the page count is much less.

Comparison

Looking at the graph below we can see three industrial sites and the pages in each of the search site’s index. When we see them side by side it’s clear Yahoo and Bing will index about the same number of pages, but they will both be far less than Google.

This trend of Bing and Yahoo not indexing all of your pages is not isolated to Industrial B2B sites.  If we look at Newegg we can see the same trend, just more pronounced.

Nothings changed over the past 3 years.  Below are some data points I took of Apple and Fidelity back in early 2011.  We can see Google’s zest for indexing the world has not dwindled at all and I don’t think it ever will.

In Conclusion

When you are looking at the health of your site’s index you should understand what the appetite of each search engine is, and know the total number of pages your site has.  Google should be willing to index most of them, but don’t be too taken back if Bing or Yahoo only indexes a 1/3 of them.