---
title: GPT-SW3 Pre-release
description: Evaluating large, generative language models for Swedish and next-generation AI applications
image: https://insights.ai.se/hubfs/1_COYm2B2zF-2Gggr0yOh2aw.webp
---

 Jan 23, 2023 11:53:00 AM

# GPT-SW3 Pre-release

![Picture of Daniel Gillblad](https://insights.ai.se/hs-fs/hubfs/daniel-gillblad-1.webp?width=50&name=daniel-gillblad-1.webp) [ Daniel Gillblad ](https://insights.ai.se/author/daniel-gillblad)

Share: [facebook-f icon ](http://www.facebook.com/share.php?u=https://insights.ai.se/gpt-sw3-pre-release) [linkedin-in icon ](http://www.linkedin.com/shareArticle?mini=true&url=https://insights.ai.se/gpt-sw3-pre-release) [Twitter icon ](https://twitter.com/intent/tweet?url=https://insights.ai.se/gpt-sw3-pre-release) [envelope icon ](mailto:?body=https://insights.ai.se/gpt-sw3-pre-release)

### Evaluating large, generative language models for Swedish and next-generation AI applications

![1_COYm2B2zF-2Gggr0yOh2aw](https://insights.ai.se/hs-fs/hubfs/1_COYm2B2zF-2Gggr0yOh2aw.webp?width=1088&height=613&name=1_COYm2B2zF-2Gggr0yOh2aw.webp)Photo by [Piotr Łaskawski](https://unsplash.com/@tot87) on [Unsplash](https://unsplash.com/photos/gL7oJLJOb_I)

Since the publication of the [Transformer architecture](https://papers.nips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html) and starting with the[ GPT](https://openai.com/blog/language-unsupervised/) and [BERT](https://aclanthology.org/N19-1423/) models in 2018, there has been a revolution in the way we model and build applications for natural language. Today, large generative language models such as variants of GPT-3 and ChatGPT show capabilities that were almost inconceivable just a few years ago. There has been an explosion of architectures and models, some publicly available and some not, but it is clear that these types of models will be of fundamental importance for AI and that models capable of representing the Swedish language will be of fundamental importance for AI applications in Sweden.

### GPT-SW3 — A generative language model for Swedish

As we have discussed in [previous posts](https://medium.com/ai-sweden/nlu/home), the GPT-SW3 initiative develops a large GPT-model for Swedish. It is driven by [AI Sweden](https://ai.se/) (the National Center for Applied AI in Sweden), together with[ RISE](https://ri.se/) (the Research Institutes of Sweden) and the [WASP WARA for Media and Language](https://wasp-sweden.org/research/research-arenas/wara-media/); these are all organizations with a main focus on promoting open research and use of AI. Our motivation for developing GPT-SW3 is to create a foundational resource for Swedish NLP that represents the entire Swedish language, and by extension, the entire Swedish population. We aim to do this as openly, transparently, and collaboratively as possible, with the goal of making the model available to all sectors in Sweden that may have a need for NLP solutions. (Further reading: “[Why do we need a large GPT for Swedish?](https://medium.com/ai-sweden/why-do-we-need-a-large-gpt-for-swedish-9a0409877f46)”

### The GPT-SW3 pre-release

While we want GPT-SW3 and similar models to be a foundational resource for everyone developing AI applications or doing research within AI, sharing such models comes with a number of challenges. Although they represent tremendous potential value, the models could possibly be used for adversarial purposes, e.g. to generate hateful, offensive, or misleading text; they may contain sensitive information and biases from the text they have been trained on; and they will require quite substantial compute resources limiting the type of users that can actually use them. However, the only way to realize the value of these models and to better understand how to mitigate these issues is to perform continued research and experimentation on real-world problems, share the learnings, and use these to continue to develop models and applications responsibly.

Therefore, we are now taking the first step in sharing the GPT-SW3 models. In this pre-release, we will share the models with those who commit to not using the models in ways that may cause harm, and who are committed to sharing the learnings of their research and applications with AI Sweden and the broader community.

**If you are interested in accessing and developing these models with us, don’t hesitate to **[**apply for access**](https://www.ai.se/en/gpt-sw3)**.**

[NLU](https://insights.ai.se/tag/nlu), [Large Language Models](https://insights.ai.se/tag/large-language-models), [GPT-SW3](https://insights.ai.se/tag/gpt-sw3), [AI](https://insights.ai.se/tag/ai)

## Related posts

[![](https://insights.ai.se/hs-fs/hubfs/1_eTudCR7cpFjknUkmat1UEQ.webp?height=200&name=1_eTudCR7cpFjknUkmat1UEQ.webp) ](https://insights.ai.se/what-is-gpt-sw3)

[NLU](https://insights.ai.se/tag/nlu), [Large Language Models](https://insights.ai.se/tag/large-language-models), [GPT-SW3](https://insights.ai.se/tag/gpt-sw3)

## [What is GPT-SW3?](https://insights.ai.se/what-is-gpt-sw3)

![Picture of Magnus Sahlgren](https://insights.ai.se/hs-fs/hubfs/Magnus_S.png?width=50&name=Magnus_S.png) [Magnus Sahlgren](https://insights.ai.se/author/magnus-sahlgren) 

 Jun 8, 2022 12:27:00 PM

A quick introduction to GPT, the NLU team at AI Sweden, and GPT-SW3. Photo by Anton Maksimov...

[ Read more ](https://insights.ai.se/what-is-gpt-sw3)

[![Prompt used: A high-resolution image depicting Scrabble tiles tumbling in the air, each spelling out the letters and symbols of _GPT-SW3_. There should be individual](https://insights.ai.se/hs-fs/hubfs/DALL%C2%B7E%202023-11-14%2016.18.28%20-%20A%20high-resolution%20image%20depicting%20Scrabble%20tiles%20tumbling%20in%20the%20air%2c%20each%20spelling%20out%20the%20letters%20and%20symbols%20of%20_GPT-SW3_.%20There%20should%20be%20individu.png?height=200&name=DALL%C2%B7E%202023-11-14%2016.18.28%20-%20A%20high-resolution%20image%20depicting%20Scrabble%20tiles%20tumbling%20in%20the%20air%2c%20each%20spelling%20out%20the%20letters%20and%20symbols%20of%20_GPT-SW3_.%20There%20should%20be%20individu.png) ](https://insights.ai.se/open-release-of-gpt-sw3)

[NLU](https://insights.ai.se/tag/nlu), [Large Language Models](https://insights.ai.se/tag/large-language-models), [GPT-SW3](https://insights.ai.se/tag/gpt-sw3)

## [Open release of GPT-SW3](https://insights.ai.se/open-release-of-gpt-sw3)

![Picture of Magnus Sahlgren](https://insights.ai.se/hs-fs/hubfs/Magnus_S.png?width=50&name=Magnus_S.png) [Magnus Sahlgren](https://insights.ai.se/author/magnus-sahlgren) 

 Nov 16, 2023 6:36:12 AM

We are now taking the next steps in the democratization and value creation of Large Language Models...

[ Read more ](https://insights.ai.se/open-release-of-gpt-sw3)

[![An image of a screen](https://insights.ai.se/hs-fs/hubfs/0_w-M1ccCWroVFqyie-1.jpg?height=200&name=0_w-M1ccCWroVFqyie-1.jpg) ](https://insights.ai.se/gpt-sw3-as-klara-the-artificial-friend)

[NLU](https://insights.ai.se/tag/nlu), [Chatbots](https://insights.ai.se/tag/chatbots), [Large Language Models](https://insights.ai.se/tag/large-language-models)

## [GPT-SW3 as Klara, the artificial friend](https://insights.ai.se/gpt-sw3-as-klara-the-artificial-friend)

![Picture of Magnus Sahlgren](https://insights.ai.se/hs-fs/hubfs/Magnus_S.png?width=50&name=Magnus_S.png) [Magnus Sahlgren](https://insights.ai.se/author/magnus-sahlgren) 

 Mar 1, 2023 11:40:00 AM

In this post, we look back at the first public display of GPT-SW3. This occurred during an art...

[ Read more ](https://insights.ai.se/gpt-sw3-as-klara-the-artificial-friend)

```json
{
  "@context" : "https://schema.org",
  "@type" : "BlogPosting",
  "author" : {
    "@type" : "Person",
    "name" : "Daniel Gillblad",
    "url" : "https://insights.ai.se/author/daniel-gillblad"
  },
  "dateModified" : "2023-10-20T07:33:34.882Z",
  "datePublished" : "2023-01-23T10:53:00.000Z",
  "headline" : "GPT-SW3 Pre-release",
  "image" : [ "https://insights.ai.se/hubfs/1_COYm2B2zF-2Gggr0yOh2aw.webp" ],
  "mainEntityOfPage" : {
    "@id" : "https://insights.ai.se/gpt-sw3-pre-release",
    "@type" : "WebPage"
  },
  "publisher" : {
    "@type" : "Organization",
    "logo" : {
      "@type" : "ImageObject",
      "url" : "https://insights.ai.se/hubfs/AI%20Sweden%20logo%20-%20yellow.svg"
    },
    "name" : "AI Sweden"
  }
}
```