×
Create a new article
Write your page title here:
We currently have 3,185 articles on s23. Type your article name above or create one of the articles listed here!



    s23
    3,185Articles



    WebSpider:

    What Is a Web Spider anyway?

    A Web spider, also known as a robot, or a worm, or a wanderer or a webcrawler, is actually a computer program that traverses the Web's hypertext structure by retrieving a document and recursively retrieving all documents that are referenced. This program retrieves information from remote sites using the standard Web protocol-Hypertext Transfer Protocol (HTTP).

    The term "spiders" or "worm," however, may give the impression that this program actually moves from one site to another and multiplies itself as it moves. These are actually some characteristics of a virus, and that is why the term "robot" could be a better name for a program that traverses the Web to retrieve information. The terms "spiders" and "robot" are used interchangeably in this article.

    Cookies help us deliver our services. By using our services, you agree to our use of cookies.
    Cookies help us deliver our services. By using our services, you agree to our use of cookies.