search engine

search engine The Internet and that part of it known as the WORLD WIDE WEB is huge and still growing (See INTERNET STATISTICS). This means that the user who wishes to find some item of information can be at a severe disadvantage. Search engines were developed in order to speed up searches within the Internet. The first search engine was known as ARCHIE. It was developed in 1990 to search the evolving Internet for FTP files. The next major advance was a tool known as VERONICA which searched GOPHER servers that contained text databases. This was followed by the program which bears the most resemblance to current search engines, the WORLD WIDE WANDERER, a primitive BOT which wandered around the Web tracking its growth. This program was controversial in that when it visited a WEB SITE it slowed down the performance of the network on which the site resided. This was followed by more sophisticated searchers such as the WWW WORM, JUMPSTATION and RSBE. The huge growth of the Web in the mid 1990s spawned a new generation of search engines that are active today, such as LYCOS, YAHOO, ALTA VISTA, and HOTBOT. A search engine is effectively an index to a large number of Web sites. Most of the current search engines provide a submission mechanism which allows the developer of a Web site to register that site with the search engine. After registration the engine automatically indexes the site URL by means of KEYWORDS. These keywords are then used in a KEYWORD SEARCH by users who wish to find some information. So in response to a query such as ‘Christmas + Card’, a search engine returns with the URL of all the sites which contain the keywords ‘Christmas’ and ‘Card’. Search engines are not just confined to Web sites. They can search for information in a wide variety of repositories and can, for example, carry out EMAIL SEARCHes and NEWSGROUP SEARCHes. Most search engines carry out automatic indexing of Web sites; although one or two rely on human indexers, YAHOO being the best known of these. Given the explosion of information on the Internet it is not surprising that search engines are able only to partially index the information that is available: most surveys put the proportion indexed on the World Wide Web at somewhere between 25 and 35 per cent of the available material. Because of this, a number of METASEARCH ENGINES have been developed. These search engines submit a search to a collection of existing search engines and collect all the responses from them. In the early days of the Internet such engines were controversial among the owners of existing search engines since they did not display the BANNER ADVERTising that was the main source of revenue of the search engine companies. Current metasearch engines include METACRAWLER and SAVVYSEARCH. The partial indexing of the Internet has also given rise to large numbers of SPECIALIZED SEARCH ENGINES. These are search engines which have links to a collection of thematically connected databases and Web sites: for example, those containing data about William Shakespeare. The vast majority of these engines consist of a number of Web pages with links to external Web sites combined with a search program running on the SERVER on which they are stored. Search engines normally use a restricted syntax for queries consisting of keywords and BOOLEAN CONNECTIVES such as AND and OR. Most of them can accept queries expressed in natural language, although the mechanism for discovering the intent of the query is usually very primitive. There are a number of specialized natural-language search engines whose interpretation of natural-language queries is more sophisticated. Probably the best known of these is ASK JEEVES. Search engines receive a huge variety of queries. A typical snapshot of a few seconds of the queries being received by Ask Jeeves included questions on the location of a map of Worcestershire, a site holding a dictionary of cat diseases, the URL of a site to download a computer game, the location of a map of the campus of an American university, the location of a site dedicated to Marilyn Monroe, and where to seek advice on detecting heart disease. See also SEARCH ENGINE SUBMISSION, MULTIPLE SEARCH ENGINE SUBMISSION, METATAG, SPAMDEX, and METACONTENT.