Why do I still see entries in the spiders report after removing "all spiders and robots" with a filter?

Why do I still see entries in the spiders report after removing "all spiders and robots" with a filter?

For products:
Webtrends On Demand
Webtrends Analytics 8.x
Webtrends Analytics 9.x

Last modified: 6/3/2009

Situation:
After applying the default "Exclude All Spiders and Robots" filter to a profile. You continue to see traffic in the Spiders report.

Solution:
There is no simple test to determine whether a visitor is a spider or not. As such, the method to identify spiders is to check each user agent (browser type) against a list of known spiders. The list of user agents treated as spiders by the Spiders report and the list used by the spiders filter are stored differently within Webtrends. Occasionally, these lists may not be in sync and spiders filter will not remove all entries treated as spiders by the spiders report.

The solution to this is to create a custom browser filter specifically excluding any spiders that still appear in the report.   

    • Related Articles

    • How do I exclude activity from common spiders and robots from my reports?

      Affected Environments: WebTrends Log Analyzer 8.X WebTrends Log Analyzer Advanced 8.X WebTrends Reporting Center 6.X WebTrends Reporting Service 6.X Solution: To filter out activity by common spiders and robots from your report, you first must create ...

    • What is a visit filter?

      Products: Webtrends On Demand Webtrends Analytics 8.0x Webtrends Analytics 8.1 Webtrends Analytics 8.5 Webtrends Analytics 8.7 Webtrends Analytics 9.2 Introduction: Unlike hit filters, which filter activity based on the occurrence of certain criteria ...

    • Why to I see "Search phrase not provided" in my search reports?

      For products: Webtrends On Demand Last modified: 7/3/2012 Situation: When looking at search reports in Webtrends On Demand or Analytics 10, the search term "Search phrase not provided" is showing up. Solution: In October 2011, Google announced that ...

    • How do I filter internal traffic based on IP address?

      For products: Webtrends Analytics 9.2a Webtrends Analytics 8.x Webtrends Enterprise 7.x Webtrends Professional 7.x Webtrends Small Business 7.x Last modified: 8/5/2010 Situation: How do I filter internal traffic based on IP address? Solution: You can ...

    • Why does my authenticated user names report say no data is available for this report?

      For version: Webtrends On Demand Webtrends Analytics 9.2x Webtrends Analytics 8.x Last modified: 1/1/2011 Situation: The Authenticated Usernames report states no data is available. Solution: Webtrends On Demand customers and software customers using ...