When thinking of artificial intelligence, most of us can’t deny that it’s a fundamental part of their life already. The unprecedented growth of artificial intelligence enables us to work more efficiently, get through everyday life easier, and do the impossible. When asking Google maps for directions, artificial intelligence is involved. When getting recommendations for new movies or shows on Amazon Prime, artificial intelligence is involved. And of course, when asking Siri about the weather tomorrow, artificial intelligence is involved.
It is no longer a far-from-the-future vision but becomes more and more important every day. Just think of trains for example; it is predicted that in a few years or decades, human train drivers are not necessary anymore as artificial intelligence will do the job. A bit intimidating if you ask me, but nevertheless artificial intelligence comes with countless benefits.
One of them is being able to generate and write texts on its own, helping humans with many tasks already. From generating content, to automated emails, letters, or even websites. Artificial intelligence is able to achieve high-quality results already which you can’t even distinguish from human-written texts sometimes.
You might’ve heard from GPT-3 (launched by OpenAI) already, an autoregressive language model that uses deep learning to create human-like texts. Right now, over 300 applications are using GPT-3 already, generating an average of 4.5 billion words every day.
Being one of the largest language models to date, GPT-3 is now competed by a new model, the so-called Jurassic-1 language model.
What is the Jurassic-1 language model?
The Jurassic-1 language model might be one of the largest AI models of its time and was launched by the Tel Aviv-based artificial intelligence development company AI21 Labs. The model works as an application programming interface (API) and is available for developers since August 2021.
What was it made for?
Yoav Shoham, the company’s co-founder and co-CEO said that it was made to enable anyone, be it publishers, students, artists, businesspeople, or researchers, to build language-based applications. You could also say that it is made available for widespread use and to challenge OpenAI’s dominance in the ”natural language processing-as-a-service” field. Language-based applications in this context means applications like virtual assistants, chatbots, content moderation, or creative writing.
AI21 Labs has been working on developing the Jurassic-1 language model since the company was founded in 2017; since then, the model has cultivated a vocabulary of 250,000 lexical items and is equipped with 178 billion parameters. As AI21 Labs says that the Jurassic-1 language model is ”slightly larger” than GPT-3, it could as aforementioned rival the GPT-3 model easily.
Both language models are able to effectively generate written language, such as emails, website layouts, or recipes for example. However, AI21 underlines that its model is more accessible to the public as it is free and can be used by anyone, who would like to try out the Jurassic-1 language model. The company’s announcement states that their mission is to fundamentally reimagine the way humans both read and write by introducing intelligent machines as strategic partners.
How does the Jurassic-1 language model work?
The Jurassic-1 language model works, similar to the GPT-3 model, with a neural network that is based on a deep learning model. It also uses existing content from websites, search engines, or other digital sources to evolve itself and become better. Other than GPT-3, the Jurassic-1 language model is not only able to generate human-like texts, but also to solve various complex tasks such as answering questions, classify texts, and many others. It is further the first model to include multi-word tokens, i.e expressions, phrases, and named entities. Due to that the Jurassic-1 language model needs fewer tokens to represent a given amount of text and can improve its computational efficiency significantly.
There a two versions of the Jurassic-1 language model available in the open beta which differ particularly by size: The J1-Jumbo model with 178 billion parameters and the J1-Large model with 7.5 billion parameters. Although it is smaller, it can work even faster but is overall less capable. But still, it is very effective for many use-cases.
Both versions can be applied to virtually any language task, the possibilities are, therefore, endless.
Jurassic-1 language model vs. GPT-3?
As the Jurassic-1 language model has 178 billion parameters, it is as mentioned before, slightly bigger than GPT-3. AI21 Lab claims that the model is the largest and most sophisticated language model ever released for general and public use.
It can recognise 250,000 lexical items, which is five times more than the capacity of all language models to date. And since these items also include multi-words, the tool has a bigger semantic representation of the human mind.
The Jurassic-1 language model was further trained with 300 billion tokens from English-language websites like Wikipedia, newspaper, StackExchange, or OpenSubtitles. Thus, it is more convenient for potential users as they can train a custom model for exclusive use with only a hand full of training examples, to be precise about 50-100 examples. AI21 Labs also claims that their model performed at par or even better than GPT-3 in a test on a benchmark suite. The performance was tested across a range of various tasks, such as answering academic and legal questions. You can tell that it is more than impressive when it comes to recognizing patterns or human language style.
However, OpenAI’s GPT-3 model has remained among the most significant artificial intelligence language models ever released. Released in May 2020, it is also able to generate impressive and unique human-like text within an eyeblink.
GPT-3 was built on 570 GB worth of data taken from the internet, including all of Wikipedia just as the Jurassic-1 language model.
It is still by far the largest known neural net created and has the capability to generate whole texts given limited context only. These texts can be any kind of text. Whether you would like to generate an essay, social media posts, translations, or even computer codes. For an intelligent machine at this scale, it’s no problem at all. It can even generate whole stories from scratch.
What might be a technological innovation for many, is seen as a threat for other. Some people are scared that artificial intelligence could become so powerful that humans become ”redundant” and are replaced by machines.
What does the future hold for the Jurassic-1 language model?
Language models can be trained to perform any language task already, and it is out of question that artificial intelligence will revolutionize the way we both read and write. The same, of course, holds true for this language model as well. Being one of the biggest models to date, the Jurassic-1 model will boost the field of ”natural languageprocessing-as-a-service” significantly and as it was made for the public to use as well, natural language generation will gain further popularity soon. Maybe you will also make us of it sooner than you might’ve thought.
