A replication of AlphaFold2 architecture (very accurate protein structure prediction model) with a free licensing.
A CLIP-like model for amino acid sequence prediction.
Github Repo: https://github.com/EleutherAI/lm-evaluation-harness
An implementation of model & data-parallel GPT-2 and GPT-3-like models with Mesh Tensorflow.
An implementation of 3D-parallel GPT-3-like models for distributed GPUs.
An enhanced version of OpenWebTextCorpus.
A large, diverse, open-source language modelling dataset.