Search Engine Research Firm IMAG Studios > Search Engine Robot

Search Engine Robot

A search engine robot, also known as a crawler or spider is a computer program used by search engines to visit other webpages and add the results it finds to the search engine database.

Search engine robots visit popular sites dozens or more times a day, and may visit less popular website infrequently perhaps once a month. Since search engine spiders typically crawl most links of a popular authortative website, having links from such a website to your own will result in your website being viewed more frequently by search engines then would otherwise occur.

The process may seem fairly straight forward, but as with most things releated to search engines and search engine optimization it tends to be more complex then at first glance.

A sample of a traditional view on search engine indexing and crawling.

It should be noted that at it's core search engines have typically evolved beyond this simple layout. Each search engine uses different methods for crawling websites with robots and spiders, and different methods for breaking down the information they find.

If the rule of thumb in business is to be seen by customers, the rule of Internet Marketing for SEO is to be seen not by search engines, but by their spiders.. and often.