The Evolution of Search
Most of inevitably use web search engines on a frequent basis to search the wide range of data on the world wide web.
Most of us inevitably use web search engines on a frequent basis to search the wide range of data on the world wide web.
Many of us also want these capabilities within the walls of our organisations, enabling us to locate the information and data we need to increase productivity, improve information reuse and locate insight we didn’t previously know about – and this is the market served by enterprise search software.
Many of these traits are goals of business intelligence (BI) products, but there are key differences – and areas where search and business intelligence are overlapping, to the extent that one day they may converge.
Typically, business intelligence tools are designed to help users looking for a specific answer or wanting to get a view of a data set, and support structured queries into known information sources – such as those containing customer or financial information. Conversely, traditional search tools are usually more free-form: Users may not know exactly what they're looking for and the data source may not be known – and the journey of finding this information can often reveal just as much useful insight as more “obvious” queries.
Accustomed as they are with the power of web search engines, business users are demanding the same search capabilities in the enterprise because they think this will allow them to search and explore business content and business data as easily as they can access information on the world wide web.
Web search and enterprise search, however, are quite different. Web search involves simple two or three word search queries that are applied to web content, content with which the user may or may not be familiar. The focus of web search is on speed, rather than accuracy, and search results are based primarily on content popularity.
Enterprise search, on the other hand, involves a much broader range of data sources, and the focus in this environment is more on accuracy than speed. Content popularity ranking systems are not appropriate for enterprise use, and this is why enterprise search tools require semantically richer interfaces involving techniques such as faceted search and natural language query processing.
Techniques used to extract metadata from business content can also be used to produce analytics from that content. This is especially the case for text data. Vendors such Microsoft, following its acquisition of FAST in early 2008, have added analytical dashboard front ends to their search tools and are marketing them as alternatives to analytical tools from traditional BI vendors. Features such as content processing, meta data extraction and faceted search are all features of the upcoming FAST Search for SharePoint 2010 release, and can be used by the FAST engine as a way of surfacing quantifiable business data that can be used in its original form, or as part of a wider BI initiative.
Over the next few years the direction of the market is clearly toward the convergence of enterprise search with business intelligence. Most major BI vendors now offer web-based portals to their products. These portals allow business users to use a search interface to locate and explore reports produced by BI processing, and to find and run canned queries and analyses.
One of the real benefits of search technology to business intelligence, however, is the ability to use this technology to access unstructured business content and convert it into a format that can be used by standard BI tools. Whereas the first generation of search tools provided indicative numbers of search results, Microsoft’s FAST technology delivers precise metrics, which can then be used to make decisions, or to provide input to other systems or services.
And as these tools evolve, and as the search engines produce more specific and quantifiable search results, the perception to end users will be that search and business intelligence solutions will be the same – after all, the end user doesn’t necessarily care what they are using, they just want results.