About spider??
Monday, January 23, 2006
Search engines use robots called " spiders " to index (categorize) your Web site. You submit your Web site and/or page to a particular search engine and the "spider" will index your submission.
Here's how it works: A spider (which is also called a "crawler") visits your Web site, reads your page... and then follows links to other pages within your site. This is what is meant when someone says your site has been "spidered" or "crawled". The spider will also return to your site on a timed and regular basis (say once every couple months) to look for any new changes.
Once the spider has all your information, it goes back home to the search engine where it will begin to "index" what it has found. Most of the time, it can take a while for new pages (or changes that the spider finds) to be added to the index. Thus, your Web page may have been "spidered", but not yet "indexed"!
Until your site has been indexed (added to the index/search engine listing), you are NOT yet available for search (you don't exist yet as far as consumers are concerned).
Now, some promotional "gurus" say that you don't have to actually "submit" your Web site/pages to these "deep" search engines because in theory , these robots are "spidering" new Web sites everyday and supposedly they will find and index your site (eventually).
But the reality is that unless you have links from other sites leading to yours, OR you have registered your site with one of these search engines, your Web site will NOT be indexed - hence, you will never exist!
posted by Nawaki's @ 6:09 PM,
