What are search technologies?
Search technologies have transformed the way we use the web. Search engines mean we don’t have to remember the URL addresses of websites or depend upon links within web pages. Web-based search programs can generally be relied on to give us the most-relevant results for a query. We enter one or more keywords to search with, and pages of results are returned to us, each result linking to a web page which we might find useful. We should seek to use search engines, such as Bing and Google, effectively and efficiently, exercising discernment in our trust of a particular page and having some grasp of the algorithms which underpin the results we get.
Why are search Technologies important?
In responding to a query, a search engine uses its own index of the web. This index is built up and maintained with specially written programs referred to as “web crawlers” (or spiders). These create a huge copy (a cache) of the publicly accessible areas of the web, stored on the search engine’s servers. When a new or updated copy of a web page is added to the cache, the search engine’s index of the web is also updated for each of the words on the page, typically ignoring small, common words like “the” and “or”. The web crawlers continue to build and update the cache by following all the hyperlinks in the page, making copies of the linked pages too, adding or updating index entries for them, and following the links on those pages as well … and so on. When we type a keyword such as “dog” into a search engine, it consults its index and returns a list of all the web pages in its cache in which that keyword appears. Typing in several keywords together (e.g. “dog” and “bowl”) can vastly narrow down a set of search results.
The really clever bit about web searches is the order in which results are ranked and displayed. ‘High quality’ web pages are considered to be ones which have lots of links pointing to them from other web pages, particularly if many of those are recognised to be high-quality themselves. On the servers of search engines, the cached and indexed copy of the (publicly accessible) web pages also includes the links between search engines. The PageRank algorithm can evaluate which pages are considered by other web developers to be of high quality. Thus, for many searches, the Wikipedia entry will top the results list simply because lots of the other high-quality search results link to it. Note therefore that this is regardless of its accuracy, authority or how many people click on it. Where applicable, the ranking of search results is personalised, taking in our location, search history and close to 200 other signals.
When teaching pupils about how search engines work, do point out the sponsored results, usually shown above or to the side of those generated by relevance algorithm. The sponsored results are adverts, themselves algorithmically generated, based on the search keyword and personalised to us in some way (e.g. taking in our previous searches). It’s in the interests of search engines to show only the adverts likely to be most relevant to us. They’re placed on a pay-per-click basis: the search engine doesn’t charge the advertiser for showing a sponsored link but it does receive income from the advertiser when a sponsored link is clicked.
Sponsored results appearing in a search for holiday deals.
A good search engine should automatically filter out explicit content. It should allow you to search within a particular website. It should enable results to be filtered by location (e.g. just the UK) and by date range (e.g. just pages created or updated during the last year). Some search engines even allow results to be filtered by reading level, restricting results to those web pages which feature shorter words or less-complex sentences.
The pieces of software we use to navigate the web are called browsers: Internet Explorer, Firefox and Google Chrome are examples. Since the web contains a vast volume of information consisting of several billion pages, search engines are a necessity to help us reach the information we ourselves require at any given time. Many people set their web browsers to display a search engine as the home page – that’s the web page that’s displayed when the browser is first opened. A web search will invariably return many pages of results, but the vast majority of searchers click a link on the first page – a sizeable portion on the very first link. This reflects our general trust that the search engine’s program will return results accurately matching the information we wish to find.
Optimising web pages so that they rank highly in search results is important for organisations seeking to maximise the number of people visiting their websites. The algorithms used by search engines are frequently tweaked to stay one step ahead of the search-engine optimisation (SEO) industry. Search engines have considerable influence over the information we access on the web – and the information we don’t get to see. This is a socio-political issue. If our pupils are to make informed choices about using search technologies, they need to appreciate the limitations of search engines and the influence of those companies developing them.