I have recently had a paper accepted by the Journal of the Association for Information Science and Technology (JASIST). This looks at the role of search literacy in the enterprise search system and its impact on search task performance. I will add the full paper reference when it is published in the Journal by Wiley later this year.

Abstract

No prior research has been identified which investigates the causal factors for workplace exploratory search task performance. The impact of user, task and environmental factors on user satisfaction and task performance was investigated through a mixed methods study with 26 experienced information professionals using enterprise search in an oil and gas enterprise. Some participants found 75% of high value items, others found none with an average of 27%. No association was found between self-reported search expertise and task performance, with a tendency for many participants to overestimate their search expertise. Successful searchers may have more accurate mental models of both search systems and the information space. Organizations may not have effective exploratory search task performance feedback loops, a lack of learning. This may be caused by management bias towards technology not capability, a lack of systems thinking. Furthermore, organizations may not ‘know’ they ‘don’t know’ (metacognition) their true level of search expertise, a lack of knowing. A metamodel is presented identifying the causal factors for workplace exploratory search task performance. Data from the Defence, Pharmaceutical and Aerospace sectors indicates the potential transferability of some of the findings.

Under the discipline of Human Computer Interaction (HCI) there have been numerous studies on the influence of the search User Interface (UI) on search behaviour. To my knowledge I am not aware of any empirical research that investigates the gathering of user feedback from the interface of an enterprise search UI. I performed some analysis recently on the impact of moving the user feedback option (button) in an enterprise search UI from the bottom to the top of the search results page. It appears to have a significant effect on how much user feedback is captured (see below).

Feedback (as a percentage of all search queries made) went from 0.2% to 0.8% an increase of 400%. True A/B testing was not applied as this was done sequentially not in parallel. However, from the feedback comments and quantitative log data, before and after the UI change there was no significant change in other enterprise search factors (e.g. indexing speed, failed searches, usage volumes). This indicates the UI change as the most likely reason for increased feedback.