Using Large Language Models to automatically create descriptions of geological images

Using Large Language Models (LLM) for automated geoscience image descriptions. There are some interesting techniques being applied to geoscience information using Frontier AI.

When a large volume of images exists, having automated techniques may help us geologists sift through it all, gain insights or pointers to potentially interesting features or areas that may warrant further investigation.

Its important we don’t overhype these techniques. Descriptions can state obvious generalities and errors. They are but another tool to potentially help geoscientists and are highly unlikely to match the abductive reasoning of a geoscientist. On the other hand, its important we don’t dismiss, or crush an idea, before its been properly tested out and iterated so it can be the best it can be. There are potental ‘sweet spots’ for such techniques that may do some heavy lifting for us when dealing with large volumes of digital information. The best approach is to try these things out and see where the most value lies as a digital assistant.

There can be misconceptions that a custom geoscience domain specific fine tuned LLM will always outperform a top ranked general purpose LLM for geoscience tasks. From my experience and other practitioners, this is absolutely not the case. Issues like catastrophic forgetting are commonplace, where a fine tuning method in a domain LLM can make one area better, but at the detriment to many others. The devil is always in the detail.

As I have stated in previous articles, always be aware of the terms and conditions of use for any free AI tool or API which is web hosted. The AI may be free but it can be at the cost of your data as you upload it, unless special arrangements are made.

A big thank you to Shivkumar Kalyanaraman of Microsoft Asia for allowing me to share these examples with the geoscience community using OpenAI’s GPT-4o. Link to Shiv’s excellent LinkedIN post: https://www.linkedin.com/posts/shivkumar-kalyanaraman-1942451_copilot-for-upstream-well-log-analysis-with-activity-7254097481909616643-ltpT?utm_source=share&utm_medium=member_desktop