A grassroots collective of researchers working to open source AI research.
GPT-J-6B, a 6 billion parameter model trained on the Pile, is now available for use with our new codebase, Mesh Transformer JAX.Mesh Transformer JAX on GitHub >
We believe the creation and open source release of a large language model is a net good to AI safety. We explain why.Why Release a Large Language Model? >
Rotary Positional Embedding (RoPE) is a new type of position encoding that unifies absolute and relative approaches. We put it to the test.Rotary Embeddings: A Relative Revolution >
GPT-Neo 1.3B and 2.7B are now available on Hugging Face Model Hub! Run the models with Transformers or call for them through their on-demand Inference API.EleutherAI on Model Hub >
GPT-Neo 1.3B and 2.7B, trained on the Pile, are now available to run with the GPT-Neo framework.GPT-Neo on GitHub >
We are proud to announce the release of the Pile, a free and publicly available 825GB dataset of diverse English text for language modeling!Visit the Pile >