Searching for information – what’s user satisfaction got to do with it?

26 Business Professionals in a multinational corporation were asked to assess their search skill prior to undertaking 2 exploratory search goal tasks (not a single right result) using their enterprise search engine. Task #1 could have potentially many results, Task #2 very few.

For each task 4 high value documents were hidden in the search index; participants were unaware. Participants had a set time based on past analysis to make it realistic, to gather together the top #10 most relevant document search results based on the simple tasks. The #4 hidden items per task would be expected to be part of these #10 gathered by participants. Participants assessed their own performance (user satisfaction) after the experiment.

  • No relationship was found between user satisfaction and objective performance.
  • Only 26% high value items were found yet 60% were satisfied/very satisfied
  • No relationship was found between self assessed search expertise and actual objective performance.
  • An association was found between relative user satisfaction (Task #2 – Task #1) and objective performance.

Those with higher search outcome performance recognised it was harder in a larger Information space to identify the most relevant results (compared to a smaller space) and adjusted their user satisfaction accordingly.

This new user satisfaction metric has a number of applications to help organisations reduce risk and increase search literacy.

https://rgu-repository.worktribe.com/OutputFile/294079

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Website Powered by WordPress.com.

Up ↑

%d bloggers like this: