A search engine is a web-based tool that enables users to locate information on the World Wide Web. Popular examples of search engines and utilize automated software applications that travel along the Web, following links from page to page, site to site. The information gathered by the spiders is used to create a searchable index of the Web.
To engineer a search engine is a challenging task. Search engines index tens to hundreds of millions of web pages involving a comparable number of distinct terms. They answer tens of millions of queries every day. Despite the importance of large-scale search engines on the web, very little academic research has been done on them. Furthermore, due to rapid advance in technology and web proliferation, creating a web search engine today is very different from three years ago.
