How to Filter out Bots and Spiders from Google Analytics

A common misconception is that Google Analytics or any other JavaScript based Web Analytics solution filters out Spiders and Bots automatically.  This was true till few years ago because most of the spiders and bots were not capable of executing JavaScript and hence were never captured by JavaScript based Web Analytics solutions. As shown in 4 reasons why your bounce rate might be wrong, these days bots and spiders can execute JavaScript and hence are showing up in your Web Analytics reports.

Google Analytics has released a new feature that will let you filter out known spiders and bots.  Here are few things to keep in mind

  1. The data will only filter spiders and bots from the day you enable this setting. It won’t be allied to the data already processed.
  2. Since this will filter out bots, you might notice a drop in your visits, page views etc.

 

Here are the steps to filter out Spiders and Bots

  1. Go to the Admin section of your Google Analytics report
  2. Click  “View” section and choose the right report view
  3. Click  on “ View Settings” (see image 1 below)
  4. Check the box under “Bot Filtering” which says “Exclude all hits from known bots and spiders” (see image 2 below)
  5. Click “Save” button at bottom and you are done.

 

filter-spider-bots-google-analytics-1Image 1

filter-spider-bots-google-analytics-2Image 2

How to Create a Filter to Tidy up Email Referrals

Roll up Yahoo Mail, Hotmail referrals

If you send marketing emails without campaign tagging or an external company e-mail your sites links to its users that have Yahoo Mail or Hotmail, users will arrive at your site via one of the many Yahoo or Hotmail servers. So searching for mail.live.com in your referral report gives you hundreds of referrers which are actually one and the same referral link.

 

 

You tend to get referrals like above ( ”xy342w.mail.live.com”). You can roll these together so you can see hotmail referrals grouped together by making changes inside Google Analytics so that reports are clearer and easier to breakdown.

 

Login to Google Analytics; select your profile you wish to roll up this data. Select Admin > Profiles > Filters

 

 

The next step is to create a New Filter.

These Filter you need to create will be a custom filter that will look at campaign sources and replace/rename the string with something that rolls them up into an easy to view source.

Filter type: Custom Filter > Search and Replace

The filter field that will be searched will be the Campaign Source and in order to ensure that all possible variables of the email source are collected a regular expression will be needed.

Filter Field > Campaign Source

Search String > “A Regular Expression”

sn144w.snt113.mail.live.com will need the following regular expression

^.*mail\.live\.co.$

Replace String with the name you wish to roll it to. I have chosen livemail.

Select Save.

Example below.

Windows Hotmail/Livemail:

 

 

This is a guest post contributed by James Cornwall

About James Cornwall

As Digital Analyst at 4Ps Marketing, James is responsible for the recently launched analytics department. After studying Civil Engineering at CITY University, James has undertaken a career in ecommerce and digital marketing. He is Google Adwords and Analytics Certified, and in his spare time is a keen hockey player. Ask James a Question: Follow on Twitter – @jamesc_4ps LinkedIn – http://uk.linkedin.com/in/james-cornwall/

Where is the Old Version of Google Analytics Reports?

Google Analytics recently switched all accounts to the new reporting interface. Earlier there used to be a link on top right hand corner to switch between old version and new version but that link is no longer there. Don’t worry the old version is still there but the link has now been moved to the bottom of the page. Not sure how long the old version will live but for now it is there. I suggest you get used to the new version as quickly as you can.

Get Google Analytics Help When You Need It

GAPremium is a Google Analytics expert consulting company.  We know Google Analytics in and out and that’s what we focus on.  If you need help with Google Analytics then we have the resources for you.

How is GAPremium Different from other Analytics Consulting Companies?

  1. We only focus on Google Analytics and related technologies only.
  2. We offer support for Google Analytics when you need and never lock you in for a long term contract
  3. Even if you need someone for few hours then we have the resources for you.
  4. We support your business 7 days a week

For Companies

  1. Get help when you need and as much you need. Minimum 1 hour commitment
  2. Get you internal technical teams trained in Google Analytics
  3. Get your business users trained in using Google Analytics to make data driven decisions

For Agencies

  1. White label our services and solutions
  2. Build you internal competency and internal team with help of our experts
  3. Tap into our resources as needed

Contact us today at support@gapremium.com for a no obligation initial consultation.

Google Analytics Campaign Tracking Made Easy

We have just released V1 of Google Analytics Campaign Tracking Sheet. This sheet makes it easy for you to keep track of the campaign your are running and also allows you to easily build a tracking URL that you can copy and paste.

Please send us your feedback so that we can improve this sheet. Feel free to tweet about this and share the love.

V2 is already in works and will be released shortly.

Download your copy of the Google Analytics Campaign Tracking Sheet

Also see How to Track Campaigns in Google Analytics

What are Filters in Google Analytics

Filters are a way to transform or manipulate the data that shows up in Google Analytics reports. Filters are applied on the data collected by Google Analytics and what you see in your reports is the final outcome after the filter has been applied to your data.

For example if you want to exclude traffic from an internal IPs then you setup an Exclude IP filter and apply it to the profile that you want to exclude the traffic from.  The final reports that you will see in that profile won’t include any traffic from those IPs.

Note: Once the filter is applied then you won’t see the original data in your reports, you will only see the transformed data.  So always apply a filter on a test profile before applying it to your main file so that you don’t accidentally mess up your reports in your main profile.

Google Analytics provides three predefined filters. If those are not the ones that you need then you also have an option to create custom filters.

Predefined filters:

  • Traffic from the domains: This filter allows you to exclude or include only the traffic from a specific domain.
  • Traffic from the IP addresses: This filter allows you exclude or include only the traffic from specified IPs.
  • Traffic to the subdirectories: This filer allows you exclude or include only the traffic to particular subdirectory on your site.

Example of a Predefined Filter

How to Exclude Internal Traffic from Google Analytics when Internal IP Address is Dynamic

Custom filters:

  • Exclude Pattern:  Allows you to exclude traffic that matches the pattern that you specify.
  • Include Pattern: This filter allows you to include only the traffic that matches the patter you specify.
  • Uppercase: Allows you to convert a given data into uppercase.
  • Lowercase: Allows you to convert a given data into lowercase.
  • Search & Replace: This allows you to manipulate the information you see in your reports by searching a value and replacing with something else. Say you have called your campaign source as Gogle instead of Google in your campaign URL. You can change it back to Google in this filter.
  • Advanced: This filter allows you to build a new data field from one or two other data fields. Some examples will be covered in future articles.

 

Examples of Custom Filters:

Regular Expressions and Google Analytics

How to Exclude Internal Traffic from Google Analytics when Internal IP Address is Dynamic

Note: Filters are applied in the Rank order, the rank 1 is applied first and then 2 and so on. To change the order use the Assign Filter Order button (see below).  Having the wrong order on filter will lead to unexpected results so make sure you have assigned the right order.