Month: May 2015

Enterprise Search – A Series of User Interfaces – not one size fits all

This is a model I have been using to discuss the concept of ‘what is enterprise search?’ with practitioners. The concept is that ‘Enterprise Search’ is supported by a series of dedicated User Interfaces (UI) shown as blue boxes, used by virtually every member of staff in the enterprise on the left, to communities of decreasing size with specific business requirements (and increasing context and complexity) as you move to the right.

Search Sketch v5

At the extreme ends denoted by the purple boxes (Company Intranet Home page for the masses on the left, niche domain application on the right), Enterprise Search has no dedicated UI per se, but is embedded within these applications, so queries are made to the Enterprise Search index ‘in context from other applications’.

The green triangles between the search index and UI is the reasoning saw tooth, representing simple traditional textual Information Retrieval (IR) needs (e.g. I need the investment proposal for asset x) and more complex natural language Question Answer (Q&A) needs that combine numerical and textual information (e.g. Who scored the first goal at the last world cup and at what minute of the game?). Critically, simple IR and complex Q&A may support both Look up and Exploratory search, hence the saw tooth across this continuum from left to right. Predictive and prescriptive analytics connect to this continuum towards the right, as prediction (what is going to happen) and prescription (what should I do about it) are just particular forms of an exploratory question.

Although the model shows a clear distinction between a dedicated Enterprise Search UI (blue boxes) and non-dedicated UI’s (Purple boxes) the difference between an Enterprise Search UI, Intranet Portal/Home Page and a domain application can become somewhat blurred, especially when canned queries (of some form) are used to deliver content (of some form) straight from the search index. However, in general the purple boxes are used to denote applications that mostly display, access and allow manipulation of content outside the enterprise search index, but have (some form) of search function.

This may help reconceptualise the debate about Enterprise Search deployments. There is always likely to be a need for the ‘common to all’ company A-Z and  ‘Google like’ page, especially for routine tasks e.g. looking up the IT Request site, booking training, claiming expenses or finding out the times of the shuttle bus from one office to another. More ad-hoc or unusual tasks such as knowing how to report phishing emails, or the travel policy to certain countries are also well served by these mechanisms, where there is typically a right answer.

However, this is unlikely to meet all the search needs of the communities within the enterprise. Firstly, is the information overload challenge. Some enterprises have tens if not hundreds of millions of items in their index likely to cause search precision issues and information overload for almost any search. The use of context (scopes) to trim the amount of information returned for particular communities through dedicated UI’s with hard coded scopes of what content in the index is to be searched, can increase search precision for communities without necessarily the need for any additional functionality, disambiguation logic or ranking algorithms. Secondly, exploratory searches (Marchionini 2006) which are open ended where there is not necessarily a ‘right answer’ require more functionality and techniques. Finding analogues for example may require components such as a spatial GIS map, table, colour coded matrix or graph using underlying knowledge representations and text analytics. Users gathering information (berrypicking) may wish to export metadata of all search results, closer to a ‘library’ report than search results page. Faceted search techniques enabling both high precision and serendipitous browsing (an Interactive Information Retrieval (IIR) technique) may be of more value to these communities regularly performing exploratory searches.

Just how many of these UI’s are needed is likely to be defined by the type of enterprise, design, size and ultimately the business requirements. Too many can be maintenance intensive, inefficient and confuse users; too few and key business needs may not be addressed. Building a network of business focal points which ‘know the enterprise’ is likely to be key to a successful deployment, as is looking for synergies across these businesses and communities. Designing for serendipity should be a principle in UI design.

Think of the Enterprise Search UI as outwardly ‘heterogeneous’ but under the bonnet is ‘homogeneously’ architected. To only cater for the high volume queries ‘what is common to all’ with a single user interface for Enterprise Search could be considered in search requirement terms, a “tyranny of the masses”.

Knowledge Organization

The role of manual & automatic tagging (classification and categorization) and information & knowledge management strategies in influencing information search and discovery in the enterprise.

A third paper has been accepted for presentation by the International Society for Knowledge Organization (ISKO) 2015 conference in London.

Theoretical Model

The best of both worlds: Highlighting the synergies of combining knowledge modelling and automated techniques to improve information search and discovery in oil and gas exploration.

Research suggests organizations across all sectors waste a significant amount of time looking for information and often fail to leverage the information they have in order to generate value and reduce risk. In response, many organizations have deployed some form of enterprise search to improve the ‘findability’ of information. Debates persist as to whether thesauri and manual indexing or automated machine learning techniques should be used to enhance discovery of information. In addition, the extent to which a Knowledge Organization System (KOS) enhances discoveries or indeed blinds us to new ones remains a moot point. Drawing on prior empirical and theoretical research, an interdisciplinary theoretical model is presented which aims to overcome the shortcomings of each approach. This synergistic model could help re-conceptualize the ‘manual’ versus ‘automatic’ debate in many enterprises, accommodating a broader range of industry needs. This may enable enterprises to develop more effective information and knowledge management strategies and ease the tension between what is often perceived as mutually exclusive competing approaches. Knowledge Organization (KO) itself may have evolved to a point where it has become difficult to distinguish it as a discrete discipline, but nevertheless plays a crucial role in a new interdisciplinary field.

Serendipity: Making your own luck

The role of the enterprise search user interface in stimulating serendipitous information encounters through presentation of the ‘unusual’.

The second paper published as part of the research study, was an experiment to stimulate serendipity in the search process through colour coded unusual word associations, tested with scientists in two organizations. It was presented at ICKM 2014 in Turkey.

Searching and browsing for unusual associations
Searching and browsing for unusual associations

Creating Sparks: Comparing Search Results Using Discriminatory Search Term Word Co-Occurrence to Facilitate Serendipity in the Enterprise

Categories or tags that appear in faceted search interfaces which are representative of an information item, rarely convey unexpected or non-obvious associated concepts buried within search results. No prior research has been identified which assesses the usefulness of discriminative search term word co-occurrence to generate facets to act as catalysts to facilitate insightful and serendipitous encounters during exploratory search. In this study, 53 scientists from two organisations interacted with semi-interactive stimuli, 74% expressing a large/moderate desire to use such techniques within their workplace. Preferences were shown for certain algorithms and colour coding. Insightful and serendipitous encounters were identified. These techniques appear to offer a significant improvement over existing approaches used within the study organisations, providing further evidence that insightful and serendipitous encounters can be facilitated in the search user interface. This research has implications for organisational learning, knowledge discovery and exploratory search interface design.
Presentation online here and Slideshare
Journal of Information and Knowledge Management (JIKM) paper here
Making Your Own Luck Article in the Geoscientist


Using local contexts to help navigate to relevant information in the enterprise search user interface.What are our information needs?

The first paper I published as a result of my studies in the Journal of Information Science, looking at what characteristics engineers prefer of search filters and why.

Enterprise Search Log
Enterprise Search Log

Retrieving Haystacks: a data driven information needs model for faceted search

The research aim was to develop an understanding of information needs characteristics for word co-occurrence based search result filters (facets). No prior research has been identified into what enterprise searchers may find useful for exploratory search and why. Various word co-occurrence techniques were applied to results from sample queries performed on industry membership content. The results were used in an international survey of fifty four practicing petroleum engineers from thirty two organizations. Subject familiarity, job role, personality and query specificity are possible causes for survey response variation. An information needs model is presented: Broad, Rich, Intriguing, Descriptive, General, Expert and Situational (BRIDGES). This may help professionals more effectively meet their information needs and stimulate new needs, improving a systems capability to facilitate serendipity. This research has implications for faceted search in enterprise search and digital library deployments.

Pre-Proof Paper Published Online Here

Journal of Information Science (JIS) Paper

Finding information – why so difficult?

This monthly blog shares thoughts from an ongoing PhD research study re-examining and re-conceptualizing searching in the enterprise; towards a conceptual model of user satisfaction and search task performance.


Despite the investments made, delivering effective enterprise search appears difficult. Business surveys indicate the average professional spends 24% of their time searching for information and approximately half of that time is unsatisfactory in some way. Some attribute this to search technology issues or governance. Others to deep underlying information and knowledge management practices, beliefs and attitudes of management. Expectations, information and search literacy of users, mental models and metacognition has also been cited as influencing factors.

Senior management such as the Chief Information Officer (CIO) may believe search capability=search technology capability. Knowledge Management and Library or Information Centres may believe search capability is a service or simply the ability for users to use specific technology systems such as the Library catalogue or a KM/lessons learnt portal.

The research will seek to shine a light on the causal factors for user satisfaction and search task performance by taking a ‘systems thinking’ approach to enterprise search.

Systems Thinking Approach

Many staff want the ‘Google Experience’ but may not understand how much relevant information may be missed even using Internet search engines. Search tasks can be broadly broken down into two categories; ‘Lookup’ search, to find known items or facts and ‘Exploratory’ open ended questions or searches for unknown quantities of information. They roughly occur in the enterprise in an 80/20 ratio respectively.

There has been little empirical research on the causes for poor user satisfaction or search task performance in the enterprise.

The research will gather data through questionnaires, interviews and focus groups from business staff, support staff and management. This will be triangulated with data from search transaction logs, feedback logs and experiments inside large and small enterprises. Data from the oil and gas, defence, space, aerospace and pharmaceutical industries will be gathered during the study.

It is hoped the research findings will help reconceptualise views towards enterprise search. If your enterprise is interested in providing data towards the research please email:

Paul H. Cleverley

Staff Page  ResearchGate Twitter LinkedIN  Google+ GoogleScholar Facebook Citeulike Slideshare YouTube Channel