AlphaFold2 Replication

In Progress

A replication of AlphaFold2 architecture (very accurate protein structure prediction model) with a free licensing.

CARP

Completed

Contrastive learning for story critiques.

CLASP

In progress

A CLIP-like model for amino acid sequence prediction.

Eval Harness

In Progress

Github Repo: https://github.com/EleutherAI/lm-evaluation-harness

GPT-Neo

Completed

An implementation of model & data-parallel autoregressive language models with Mesh Tensorflow for distributed TPUs.

GPT-NeoX

In Progress

An implementation of 3D-parallel autoregressive language models for distributed GPUs.

Mesh Transformer JAX

Completed

An implementation of model & data-parallel autoregressive language models with JAX and Haiku for distributed TPUs.

OpenWebText2

Completed

An enhanced version of OpenWebTextCorpus.

The Pile

Completed

A large, diverse, open-source language modelling dataset.