IMAG Studios SEO > Search Engine Spider

List of Search Engine Spiders

Search Engine Spiders as you know are used by the search engines to view websites that are already in their index, or that they are crawling with plans to potentially index. A spider will crawl the links on a page it discovers, the more authority that page has th e deeper it will crawl and the more links it will discover.

If you check your server logs, you'll get an indication of what search engines have discovered your website naturally. Most search engine spiders will leave their trail clearly for you to see, but their spiders names don't always show you clearly what search engine they come from. Here is a list of Search Engine spiders and their home search engine.

Euroseek Search Engine : Arachnoidea

Excite Search Engine: ArchitextSpider (no longer active)

Deepindex Engine: Deepindex

Alltheweb Engine: FAST-WebCrawler

SearchHippo Engine: Fluffy the spider

Google Search Engine: Googlebot

Gigablast Search Engine: Gigabot

Northernlight Engine: Gulliver (inactive today)

Yuntis Engine: Gulper

Alexa Search Engine: ia_archiver

Lycos Search Engine: Lycos_Spider or T-Rex (No longer active)

LookSmart Search Engine: MantraAgent

MSN Live Search Engine: MSN

Websmostlinked Engine: Nazilla

Openfind Search Engine: Openbot

Altavista Search Engine: Scooter

Scrub The Web: Scrubby

Inktomi Search: Slurp

Teoma Search Engine: Teoma_agent1

WebCrawler Search Engine: WebCrawler