If your like me, and you should be, you’re diligent about checking your website statistics to see the flow and trends of visitors coming to your site, all this data will help you fine tune your web efforts.

All sorts of insightful information can be obtained when viewing your website statistics including unique vs returning visitors, browser types, monitor resolutions, search terms, how visitors arrived at your site and on and on. If you haven’t spent some time looking through your server stats I encourage you to do so.

There are typically two types of server statistics you’ll use to provide you this information Server Logs and Google Analytics, there are others but these are really the only two I’d bother with.

Let me explain briefly the difference between the two.

Server Logs and Server Log Readers
Most reputable web hosting companies provide an hosting administration tool for managing your online account, I recommend finding a host that uses cPanel. cPanel gives you total control over managing every aspect of your hosting account including the ability to view your website statistics through an installed server applications like Webalizer and AWStats, both of these programs read your raw server logs and present it to you in an easy to understand manner. More information on web hosting and cpanel can be found in an earlier blog post entitled “Web 101 – Back to the Basics Part 2 Web Hosting.”

Webalizer and AWStats are both very good at deciphering server logs and will both give you valuable insight into your site traffic.

Google Analytics
Google analytics is a hosted service offered by the folks at Google that will give you all the data you ever wanted in regards to to website traffic and trends. Better than me trying to explain every single feature of the Google Analytic service, I suggest jumping over to http://www.google.com/analytics/features.html and reading through the features list, once you’ve finished this post of course.

Though beyond the scope of this post, integration of Google Analytics into your existing website is relatively easy and painless. Where Google Analytics really shines is in it’s ability to search a specific time frame and allowing you to view all your data in colorful bar graphs and pie charts which make it very easy to do month to month comparisons, reports can even be emailed in PDF format.

I’m constantly asked by clients is “How come my Webalizer and AWStats vary drastically from my Google Analytics Stats?” The answer is simple. Google Analytics does not count Spiders or Robots and AWStats and Webalizer does, period! I know your thinking to yourself I thought you said the answer was easy so what the heck is a Spider or Robot? Let me explain.

Spiders / Bots / Crawlers
Are all names for web based applications that follow links on your site with the sole intent of gathering information. There are good bots and spiders such as the googlebot that indexes your site to help you in your placement of google search rankings while bad bots are used to harvest email addresses and personal data off your website.

Bots and spiders can be instructed to not follow specific links by including a simple robots.txt text file at the root level of your server. The easiest way to create a robots.txt file is to again head on over to Google and sign up for a Webmaster Tools account https://www.google.com/webmasters/tools/docs/en/about.html along with a host of great tools they also offer a robot.txt generator. Plug in your parameters and then upload the generated file to your web server via FTP and your all set to go

So if you’re wondering why your stats don’t match try placing a robots.txt file on your server and send those Robots packing.