The Basics: How Search Engines Work?

First, let’s look at how crawler-based search engines work (both Google and Yahoo fall in this category). Each search engine has its own automated program called a “web spider” or “web crawler” that crawls the web. The main purpose of the spider is to crawl web pages, read and collect the content, and follow the links (both internal and external). The spider then deposits the information collected into the search engine’s database called the index.

When searchers enter a query in the search box of a search engine, the search engine’s job is to find the most relevant results to the query by matching the search query to the information in its index.

What makes or breaks a search engine is how well it answers your question when you perform a search. That’s based on what’s called the search engine algorithm which is basically a bunch of factors that the search engine uses to say “hey is this page RELEVANT or NOT?”. The higher your page ranks for these factors (yes some factors are more important than others) than the higher your page will get displayed in the search engine result pages.