We started from the printed index to The Times which I think was issued quarterly, and card indexes in local libraries. In recent decades we have moved from conventional indexes which are constructed using a controlled list of index terms, to free-text searching where almost any word will serve as a search term but may give results you were not expecting e.g. synonyms. When computers were first applied to the problem their capacity was not sufficient to support this kind of searching so terminology was still controlled. The earliest on-line databases were largely in the field of science which lends itself to that. Free searching of such a huge newspaper database has its pitfalls.
This change may actually be the result of some analysis about how people have used the index, and how long it has taken each search to find a satisfactory result. What the new style is trying to do is guide your choice of search terms within some broad categories, and organise your thoughts, so slightly more structured than a free-for-all, to give better results overall.
