EleutherAI

Announcing GPT-NeoX-20B, a 20 billion parameter model trained in collaboration with CoreWeave.

GPT-J-6B, a 6 billion parameter model trained on the Pile, is now available for use with our new codebase, Mesh Transformer JAX.

We believe the creation and open source release of a large language model is a net good to AI safety. We explain why.

Rotary Positional Embedding (RoPE) is a new type of position encoding that unifies absolute and relative approaches. We put it to the test.

GPT-Neo 1.3B and 2.7B are now available on Hugging Face Model Hub! Run the models with Transformers or call for them through their on-demand Inference API.

GPT-Neo 1.3B and 2.7B, trained on the Pile, are now available to run with the GPT-Neo framework.

We are proud to announce the release of the Pile, a free and publicly available 825GB dataset of diverse English text for language modeling!