All search engines contain the following main components:
| Spider | A browser-like programme that downloads web pages |
| Crawler | A program that automatically follows all of the links on each web page |
| Indexer | A program that analyzes web pages downloaded by the spider and the crawler |
| Database | Storage for downloaded and processed pages |
| Results engine | Extracts search results from the database |
| Web server | A server that is responsible for interaction between the user and other search engine components |