Can we improve corporate search?

I had an article published in Digital Energy Journal this month. Link here

Most of us expect a search engine to be a tool which delivers us results such as documents, web pages, people profiles, lessons learnt and best practices when we type something into a box. But can it do more? Oil and gas data scientist Paul Cleverley is doing a PhD to try to find out.

Development of ‘enterprise search’ technology is fairly stagnated in companies, said Paul Cleverley, speaking at the Digital Energy Journal Aberdeen conference in May, ‘Subsurface computing and competitive advantage’.

It is usually seen as a utility, not something which affects the bottom line of the company. ‘Quite often the user interface is a pretty bland search box,’ he said. This may follow the theory-in-use dominant culture of Google which drives our expectations, but are some latent needs going unmet?

Considering the benefit to the company of making it easier for people to find what they are looking for and what may be valuable, perhaps it is worth investing in a better search engine architecture and design, Mr Cleverley believes.

Company information as a whole may be under-exploited and under-explored. Where the ‘whole is greater than the sum of the parts’, using the right approach, company information may be able to surface an answer or association that is not present in any one single document. Using a common metaphor, as well as finding ‘needles in haystacks’, smashing together information haystacks and finding ‘new needles’ could be a gamechanger.

This isn’t just an oil and gas problem. Scientists and engineers are generally interested in similar concepts in any industry, he said. Some of the concepts from the research have been shared with NASA and incorporated into their communication and designs.

Better searching tools can help geoscientists be more objective – weighing up a range of different possibilities, rather than sorting for information which fits their hypothesis.

Many of us may have heard executives saying that having a geologist that knows the basin inside out is a valuable asset – but can also be a liability, if he or she is not willing to engage with an alternate point of view about how it works.

Paul Cleverley is an information scientist. He is in his 4th year of a PhD at Robert Gordon University, Aberdeen…



The Modality of Search (Take 2)

After presenting at conferences in the US and Europe, I have made some minor modifications to the ‘Modality of Search’ model I posted a few months ago. Version 2 below. This proposes a new way in which to view Enterprise Search and Discovery Capability.

Link to PowerPoint and Explanation in SlideShare Click Here

It is early days, but there is some evidence the model and narrative is capable of changing the mind-sets of senior executives, leading to a different strategy and investment approach being taken towards ‘search’ within the organization.




Delighted to be appointed to the Board of Directors of GeoScienceWorld (GSW) in the capacity of researcher-at-large last month. GeoScienceWorld is a non-profit collaborative for research and communications in the earth sciences.

GeoScienceWorld was founded by the American Association of Petroleum Geologists • American Geosciences Institute • Geological Society of America • Geological Society of London • Mineralogical Society of America • Society for Sedimentary Geology • Society of Exploration Geophysicists. More here:


Teaching machines about a subject like oil and gas

Many organizations are sitting on a wealth of unstructured text. There are many OpenSource and free tools than can help build large scale associative networks in either unsupervised or semi-supervised ways.

With exponentially increasing volumes of information, much information is being ranked or suggested by popularity. That may effectively ‘censor’ some information through its obscurity. With an increasing need/intent for search engines to ‘show me something I don’t already know’ there appears a need to revisit ‘relevance’ algorithms. What is most popular, is not necessarily what is most interesting.

Search engines may be increasingly the way in which ‘we come to know’. If a corpus is the starting point, allowing the user to explore associative networks in various ways (not just by popularity), as a series of click-able facets, may mitigate the issues presented with a classic search box, where a searcher may be hampered to find out what they don’t know by their own existing knowledge of keywords. In other words, the agency of the searcher using traditional search engines may limit their ability to discover something they have no advance knowledge of. Exploiting associative networks may be a useful way of discovering new knowledge.

More here at LinkedIn:

Slides and references: