The Dawn of a New Era in Natural Language Understanding and Generation

When I look back to 1984 when I received my 32k home computer, at the time I didn’t think I was at the dawn of personal computing and an explosion of creativity.

The more I delve into the 175Billion parameter (in AI size matters) Transformer GPT-3 Language Model built from vast swathes of the Internet, the more I think this is the dawn of a new era. How we can build apps to understand and generate natural language to do many things maybe we don’t even realise yet.

Augmented creativity.

It may make the current crop of stuff we use around ‘help assistants’, chatbots, conversational assistants, text summarisation etc look like a ZX81…

The CEO of OpenAI has been reported as saying, “The GPT-3 Hype is too much. AI is going to change the world, but GPT-3 is just an early glimpse”.

I’m a natural sceptic. But I have been taken aback after seeing what possibilities GPT-3 opens up. Let’s be clear, this is not true artificial intelligence, in many ways the machine is still dumb. This has not changed. At the same time I don’t want to downplay the practical advances made and direction of travel.

I’ll leave other sources to discuss the ethics challenges (I mentioned these in a post I made in June 2019), Microsoft’s exclusive licensing of the source code and concerns in the research community.

Focusing on use cases, some of these demonstrations are eye opening:

https://github.com/elyase/awesome-gpt3

https://towardsdatascience.com/gpt-3-demos-use-cases-implications-77f86e540dc1

https://machinelearningtokyo.com/2020/07/26/10-cool-gpt-3-demos/

From writing creative fiction, creating designs, automatically writing Python code from plain English instructions, generating website designs, creating charts from descriptions, writing like a lawyer to simplifying legal language, automatically generating emails from just a few bullets, question & answer and reasoning.

It is still technology and it’s up to us to decide how we can use it for the benefit of science, business and society. I’m very excited about how these can be applied to the geosciences.