top of page

Trondheim’s role in the GPT revolution

Rolf Dyrnes Svendsen, NorwAI’s communication manager and former editor at Adresseavisen, digs into the impact of AI on text generation and how Trondheim and the Norwegian language can be part of this new future.


Inside NTNU's super computer (Photo: Kai T. Dragland)

Right now, Norway’s most powerful supercomputers are running around the clock to create an essential building block for the country’s technological future — and it’s all happening right here in Trondheim.


NTNU’s supercomputers are putting together a Norwegian language GPT model with 20 billion parameters. To give you an idea of just how big that is, it’ll require the supercomputers’ entire processing power for whole two months to create it. But what is the purpose of this model? And why is it so important for Norway?


AI is here

Silicon Valley's ChatGPT is taking the world by storm. Created by OpenAI — which is backed by Microsoft and Elon Musk among others — the chatbot has incredible conversational skills. The possibilities shown by this technology have made it clear to anyone how artificial intelligence can contribute to their work and life. But while ChatGPT is great in English, it’s not so impressive in Norwegian. The supply of text material to train AIs is much greater in English than in smaller languages, which means language models gradually become worse for less-used languages.


The race to develop the biggest and best language models is currently going on between the great powers in San Francisco and Beijing — but Norway has its own small horse here in Trondheim. Large language models can potentially transform industries like the internet and the smartphone did in the past. The models provide new content, knowledge and creativity with AI-based applications. Several challenges remain to be solved, but the language models will accelerate productivity and growth.


If other language communities are not to become completely dependent on English or Chinese-based technology, they must develop their own language models to safeguard their digital independence, competence, and competitiveness. The EU has 24 official languages. Norwegian is not one of them.


Norwegian alternative

The research centre NorwAI at NTNU has already developed two smaller GPT test models since the centre opened two years ago. Now, at the start of 2023, the centre is preparing for a big leap. NorwAI will have its model ready later this spring, based on Norwegian input, among it data from the media group Schibsted. The partners in the NorwAI consortium will be able to play-test the model for their purposes. The next step will be to build a twice as large version of 40 billion parameters before expanding further.


BIG, BIGGER, BIGGEST: Parameters express how fine nuances are captured in the language's orthography, syntax, semantics, and pragmatics. (This spread originally appeared in the Tech List section of The List's 2023 Spring issue)

GPT stands for ‘Generative Pretrained Transformers’ and builds on artificial intelligence to learn and understand the input. Billions of words and information from online and lexical sources are processed. In short, the large language models' basic functionality is to predict the next word in a context via probability calculations. This enables impressive language solutions that generate texts for self-selected topics, summarize complex matters, answer questions in ongoing dialogues, etc. Language models are also capable of producing speech and illustrations.


Will affect us all

Throughout history, only humans have been able to express complex prose creatively. Now the large language models (LLM) show astonishing results. They will reduce the price of knowledge and creativity. They will impact all creative and knowledge-based professions, and they work as text-to-text, speech-to-text, and text/speech-to-illustrations.


They will affect our relationship with machines. Professions will change, some will be replaced, and new ones created. The new profession of AI-operator will be born. AI is entering education, sales and marketing, and professions such as lawyers, journalists, programmers, designers, etc. will be affected. The reason is simple. In creative subjects, mistakes are cheap and acceptable because people are involved in the process anyway.


There is still a lot of development work to be done for large language models. They do not always distinguish between fact and fiction, they hallucinate. They are language models, not sources of knowledge. Verifying language models will become a major area of research. But improvements will come — and Trondheim will be a part of them.



This article first appeared in the Tech List section of The List's Spring 2023 magazine — which is made possible by Trondheim Tech Port.



Comentários


bottom of page