One of the disadvantages of Google Analytics is that it can’t track search engine spiders/crawlers activity. In other word, when GoogleBot, Yahoo! Slurp, MSNBot, Bing and other major search engine crawlers visit your website, the data will not be logged by Google Analytics. To get a detail view of these crawlers activity of your site, you need CrawlTrack, an open-source PHP spider and crawler tracker web analysis script for webmasters. It gives gives you individual crawler stats for each page of the site crawled, as well as stats into what percentage of your site pages were not crawled. The data of crawler’s activity is very useful for SEO, since when you publish new content on your site, you are able to monitor exactly when and how often this new content is indexed by search engines.

Strong Functionality of CrawlTrack

• Web site statistics, giving number and origin of visitors like referrer, key word used, country of origin pages viewed and bounce rate;
• Crawler visits statistics, hits number and pages viewed by search engine crawlers;
• Indexation in the main search engines follow-up like backlinks and indexed pages;
• Record and blocage of certain type of hacking attempts like code injection or SQL injection;
• Download counter;
• Record of number and origine of 404 errors.
• http://www.crawltrack.net/

Installation
The script of CrawlTrack is installed on your own server, so you can keep control of the data. All you need to do is upload the files to a new directory on your hosting account, create a new database on your MySQL server, include a piece of CrawlTrack PHP code in your website code and go through the automated installation procedure.