Projects | EleutherAI

#theme-toggle, .top-link { display: none; } @media (prefers-color-scheme: dark) { :root { --theme: #1d1e20; --entry: #2e2e33; --primary: rgba(255, 255, 255, 0.84); --secondary: rgba(255, 255, 255, 0.56); --tertiary: rgba(255, 255, 255, 0.16); --content: rgba(255, 255, 255, 0.74); --hljs-bg: #2e2e33; --code-bg: #37383e; --border: #333; } .list { background: var(--theme); } .list:not(.dark)::-webkit-scrollbar-track { background: 0 0; } .list:not(.dark)::-webkit-scrollbar-thumb { border-color: var(--theme); } }

AlphaFold2 Replication

In Progress

A replication of AlphaFold2 architecture (very accurate protein structure prediction model) with a free licensing.

CLASP - Contrastive Language-Amino Acid Sequence Pretraining

In progress

A CLIP-like model for amino acid sequence prediction.

Eval Harness

In Progress

Github Repo: https://github.com/EleutherAI/lm-evaluation-harness

GPT-Neo

Completed

An implementation of model & data-parallel GPT-2 and GPT-3-like models with Mesh Tensorflow.

GPT-NeoX

In Progress

An implementation of 3D-parallel GPT⁠-⁠3-like models for distributed GPUs.

OpenWebText2

Completed

An enhanced version of OpenWebTextCorpus.

The Pile

Completed

A large, diverse, open-source language modelling dataset.