Open Language Model from Allen Institute for AI

The Allen Institute for AI is building OLMo (Open Language Model):

AI2 is embarking on the creation of an open, state-of-the-art generative language model: AI2 OLMo (Open Language Model). OLMo will be comparable in scale to other state-of-the-art large language models at 70 billion parameters, and is expected in early 2024.

OLMo will be a uniquely open language model intended to benefit the research community by providing access and education around all aspects of model creation. OLMo will be a new avenue for many people in the AI research community to work directly on language models for the first time. We will be making all elements of the OLMo project accessible — not only will our data be available, but so will the code used to create the data. We will release the model, the training code, the training curves, and evaluation benchmarks. We will also openly share and discuss the ethical and educational considerations around the creation of this model to help guide the understanding and responsible development of language modeling technology.

This broad availability of all aspects of OLMo will allow the research community to directly take what we create and work to improve it. We believe that millions of people want to better understand and engage with language models, and we aim to create an environment where they actually can, leading to faster and safer progress for everyone. Our goal is to collaboratively build the best open language model in the world.

There's more at the link.