Index to Crawl Ratio

Posted On August 01st, 2014 by admin

Index to Crawl Ratio:  A Key SEO Performance Indicator

For Search engines, it is literally true that they must learn to crawl before they can index your site.  Crawling or spidering your site is the process of a search engine robot or bot finding a web page on the Internet and then caching the content to be indexed by the search engine.  If a spider cannot crawl your site it will of course not be able to index, categorize and store its content using its algorithm.  There are many variables that can affect the crawling of your site such as a slow server response to http requests and broken links with page not found.

Most commercial engines have some tool or report that you can use to assess if and what pages are being crawled over a given time period.  You can also look at your server logs to view search engine access activity for specific pages.  Once you determine that your site is being crawled, the next logical question is what information is it finding and how is it indexing that information for web searches.  The ratio of the number of pages indexed to the number of pages crawled is called the Index to Crawl Ratio usually represented as the ratio of X:Y where X is the number of indexed pages and Y is number of pages crawled per domain.
To calculate this in Google, you can go to their webmaster tools and under the statistics for your account you will find:  Index Stats and Crawl Stats.  Find the total number of pages indexed divided by the average number of pages crawled to get the ratio for your site.  This number will provide a good indication of the overall health of your site's indexability over time.  The higher the ratio the more information Google is deciding to index per page.  Monitor this number to find patterns and measure success of your SEO program.