I just read a report that said that most people don’t know how search engines work (l). That just isn’t right. You need to know how a search engine works for three reasons: 1) it helps you be better searchers, 2) it helps you make your site easier to find, and (3) I think it’s neat. So this week I’m going to talk about search engines. Here are some things you should know:

• No search engines indexes the entire web. In fact, even together they don’t index the entire web. Information that isn’t indexed includes new web sites, databases on web sites, web sites that no one links to, and web sites where the information changes often (the site is probably indexed the dynamic information isn’t).
• People pay to get their sites to come up in your search. People buy “keyword ads” so that every time you do a search on the term they buy their site will come up near the top of the list. Some search engines do a better job of others at distinguishing between the paid results and the actual search results.
• While there aren’t really any “tricks” for getting your web site to the top of the list (except through keyword ads) there are some standards to follow to improve your chances. The sites that know these standards do not necessarily have the best information – so sometimes it helps to look beyond the first page of results when you do a search.
• With most search engines the popular sites will get to the top of any search. Popularity is defined either by the traffic a site gets or by the number of sites that link to a site. Sites that are popular are not always the best, so again look beyond the first page of results.
• Some search engines are limited by the type of format they can track. Some include PDF, Word, PowerPoint, and other documents in their search; others search only html (web page) documents.

Search engines (such as Google) index the words on a web site. They use a “robot” or “spider” to track the number of times a web site mentions a particular word, what words are used near each other, and where on the document a word is found. (They also factor in a few other details.) The spiders go from one web site to another through the links on a page; that is how they grow their indices. They help you find what you need by matching your search terms with the words in their database. Sometimes is works well; other times it doesn’t.

If you want to know more about how a search engine works I suggest the How Stuff Works page!