This dataset bundle contains three sets: 1) domain sequences for pretraining, 2) domain sequences for finetuning and 3) variant fitness scores. Files are in lmdb format. 1. Domain sequences for pretraining Two bz2 compressed files are provided: rp15_seq_lmdb.tar.bz2: representative proteome sequences at 15% level from Pfam-V32 database. Whole dataset is randomly split into train and validation sets: number of sequences in training set: 12,681,738; number of sequences in validation set: 1,042,103. Sequence length range from 18 to 500 (inclusive) and this length filtered set covers more than 95% sequences of the whole set. rp75_seq_lmdb.tar.bz2: representative proteome sequences at 75% level from Pfam-V32 database. Whole dataset is randomly split into train and validation sets: number of sequences in training set: 68,810,960; number of sequences in validation set: 5,687,282. Sequence length range from 18 to 500 (inclusive) and this length filtered set covers more than 95% sequences of the whole set. Information of each sequence is stored as key-value pairs: { 'primary': protein amino acid sequence, 'protein_length': length of the sequence, 'family': sequence Pfam family id (without 'PF'), 'clan': sequence Pfam clan id (without 'CL', -1 if not exists), 'unpIden': sequence Uniprot_id.version_number, 'range': domain residue start-end indices (follow indices of Uniprot seq), 'id': a index number for each sequence from 0 to N } One example: {'primary': 'ALQTTDKHHVATPANWRPGDDVIVPPPATQEAAEERLREG', 'protein_length': 40, 'family': 10417, 'clan': -1, 'unpIden': 'A0A147JSN0.1', 'range': '162-201', 'id': '0'} 2. Domain sequences for finetuning We collected homologous sequences of 33 proteins from [Shin2021]. The sequences are domain sequences queried over UniRef100 database. Each family is split into train and validation sets with ratio 9:1 Information of each sequence is stored as key-value pairs: { 'unp_range': Uniprot record name/start index - end index (indices follow Uniprot seq), 'primary': protein amino acid sequence, 'seq_reweight': sequence weighting score from Shin2021, 'family_reweight': family weighting score from Shin2021 (sum of seq_reweight score for all family sequences), 'seq_reweight_mmseqs2': sequence weighting score calculated by us using mmseqs2, 'family_reweight_mmseqs2': family weighting score based on seq_reweight_mmseqs2 (sum of seq_reweight_mmseqs2 score for all family sequences) } One example: { 'unp_range': 'AMIE_PSEAE/1-346', 'primary': 'MRHGDISSSNDTVGVAVVNYKMPRLHTAAEVLDNARKIAEMIVGMKQGLPGMDLVVFPEYSLQGIMYDPAEMMETAVAIPGEETEIFSRACRKANVWGVFSLTGERHEEHPRKAPYNTLVLIDNNGEIVQKYRKIIPWCPIEGWYPGGQTYVSEGPKGMKISLIICDDGNYPEIWRDCAMKGAELIVRCQGYMYPAKDQQVMMAKAMAWANNCYVAVANAAGFDGVYSYFGHSAIIGFDGRTLGECGEEEMGIQYAQLSLSQIRDARANDQSQNHLFKILHRGYSGLQASGDGDRGLAECPFEFYRTWVTDAEKARENVERLTRSTTGVAQCPVGRLPYEGLEKEA', 'seq_reweight': 0.0714285714286, 'family_reweight': 19553.99941694187, 'seq_reweight_mmseqs2': 0.0021413276231263384, 'family_reweight_mmseqs2': 25236.560885598774 } 3. Variant fitness scores This fitness benchmark set contains 42 mutagenesis sets, which were from originally curated by [DeepSequence] and later [Shin2021] used a subset of it. Information of each variant is stored as key-value pairs: { 'set_nm': set name, 'wt_seq': WT sequence, 'seq_len': sequence length, 'mutants': amino acid variants list (could have multi-site mutations), 'mut_relative_idxs': list of relative amino acid indices for variants, 'mut_seq': mutant sequence, 'fitness': fitness score } One example: { 'set_nm': 'AMIE_PSEAE_Whitehead', 'wt_seq': 'MRHGDISSSNDTVGVAVVNYKMPRLHTAAEVLDNARKIAEMIVGMKQGLPGMDLVVFPEYSLQGIMYDPAEMMETAVAIPGEETEIFSRACRKANVWGVFSLTGERHEEHPRKAPYNTLVLIDNNGEIVQKYRKIIPWCPIEGWYPGGQTYVSEGPKGMKISLIICDDGNYPEIWRDCAMKGAELIVRCQGYMYPAKDQQVMMAKAMAWANNCYVAVANAAGFDGVYSYFGHSAIIGFDGRTLGECGEEEMGIQYAQLSLSQIRDARANDQSQNHLFKILHRGYSGLQASGDGDRGLAECPFEFYRTWVTDAEKARENVERLTRSTTGVAQCPVGRLPYEG', 'seq_len': 341, 'mutants': ['M1W'], 'mut_relative_idxs': [0], 'mut_seq': 'WRHGDISSSNDTVGVAVVNYKMPRLHTAAEVLDNARKIAEMIVGMKQGLPGMDLVVFPEYSLQGIMYDPAEMMETAVAIPGEETEIFSRACRKANVWGVFSLTGERHEEHPRKAPYNTLVLIDNNGEIVQKYRKIIPWCPIEGWYPGGQTYVSEGPKGMKISLIICDDGNYPEIWRDCAMKGAELIVRCQGYMYPAKDQQVMMAKAMAWANNCYVAVANAAGFDGVYSYFGHSAIIGFDGRTLGECGEEEMGIQYAQLSLSQIRDARANDQSQNHLFKILHRGYSGLQASGDGDRGLAECPFEFYRTWVTDAEKARENVERLTRSTTGVAQCPVGRLPYEG', 'fitness': -0.5174 } Reference DeepSequence: Riesselman, Adam J., John B. Ingraham, and Debora S. Marks. "Deep generative models of genetic variation capture the effects of mutations." Nature methods 15.10 (2018): 816-822. Shin2021:Shin, Jung-Eun, et al. "Protein design and variant prediction using autoregressive generative models." Nature communications 12.1 (2021): 1-11.