A highly favourable item on a new search engine, blinkx in the Guardian Online supplement, sent me off to its Website to check it out. blinkx uses, so we are told, an AI technique rather than page-ranking a la Google and it searches not only the Web, news services, and Weblogs, but also your hard disc. From one of the file names on the downloaded system I suspect that the engine behind blinkx is Autonomy
The Website includes an option to try out the beta version of blinkx optimised for broadband users and I discovered something rather odd. The PR claims that "blinkx understands your question and presents you with links as you search." - but the system obviously uses stop words. How can a question be understood if the stop words include terms of significance to the user?
Specifically, I searched for 'Information Research', expecting the journal site to pop up fairly quickly - no: only things on 'research' appeared. Similarly, when I used 'information behaviour', only 'behaviour' was used as a search term, and for 'information science', only 'science'. Not much use in the information management sector, then! The give-away is that the terms used in the search are highlighted and in all cases, where 'information' did appear in an item, it was not highlighted.
'Information' on its own may or may not be a useful search term - certainly it would generate millions of hits, but when used in compounds such as those mentioned, the concept so formed has much greater specificity. As long as AI systems continue to fail to recognize concepts and their semantic significance, they will fail to produce a search system that is a significant improvement on Google.