SEO: detect search engines robots

With power of flightPATH it's possible to detect spiders visits.
There are number of advantages to have that ability.

SEO masters can log such visits and use that info for SEO statistics.

SEO Masters may need to be notified about spiders visits immediately by e-mail.

Another powerful ability is you can provide different page to spider than to end user.
This page may have content optimised for Search Engines.

Following are sample rules:

[jetnexusdaemon-Path-15]
Desc="Log spider requests of popular search engines"
Name="Log Spider"

[jetnexusdaemon-Path-15-Condition-1]
Check="match"
Condition="header"
Match="User-Agent"
Sense="does"
Value="Gogglebot|Slurp|msnbot|ia_archiver"

[jetnexusdaemon-Path-15-Evaluate-4]
Detail="User-Agent"
Source="header"
Variable="$crawler$"

[jetnexusdaemon-Path-15-Function-1]
Action="log"
Target="[$crawler$] $host$$path$?$querystring$"

[jetnexusdaemon-Path-23]
Desc="Give a spider different page"
Name="Cheat Spider"

[jetnexusdaemon-Path-23-Condition-1]
Check="match"
Condition="header"
Match="User-Agent"
Sense="does"
Value="Gogglebot|Slurp|msnbot|ia_archiver"

[jetnexusdaemon-Path-23-Function-1]
Action="rewrite_path"
Target="$path$-seo.html"



Edited by softscrape - 17 Mar 2011 at 7:29am

It's OK, I worked it out.