These files contain results from running heldout estimation on the CIFAR-10N dataset with the rand1 noise type, using 1500 ResNet models. The cifar-rand1-human-infl-mem.npz file contains results for human noisy labels, while cifar-rand1-syn-infl-mem.npz contains results for synthetic noisy labels generated using the same class transition matrix. Dictionary Keys 1. total_runs Type: Integer Description: The total number of training runs included in this aggregated result. 2. trainset_mask Type: array (Boolean, shape: [total_runs, train_size]) Description: A mask indicating which training examples were used (True) or held out (False) during each training run. 3. trainset_correctness Key: trainset_correctness Type: array (Boolean, shape: [total_runs, train_size]) Description: Whether the model correctly predicted the label for each training example during each run. 4. trainset_predictions Type: array (Integer, shape: [total_runs, train_size]) Description: The predicted class labels for each training example during each run. 5. testset_correctness Type: array (Boolean, shape: [total_runs, test_size]) Description: Whether the model correctly predicted the label for each test example during each run. 6. testset_predictions Type: array (Integer, shape: [total_runs, test_size]) Description: The predicted class labels for each test example during each run. 7. memorization Type: array (Float, shape: [train_size]) Description: Memorization score of each training example, computed across all runs. 8. influence Type: array (Float, shape: [test_size, train_size]) Description: Influence scores of each training example on each test example. 9. memorization_inclusion_prob Type: array (Float, shape: [train_size]) Description: Probability that the training example is predicted correctly when included. 10. memorization_exclusion_prob Type: array (Float, shape: [train_size]) Description: Probability that the training example is predicted correctly when excluded. Usage To load the file and access its contents: import numpy as np # Load the file data = np.load('cifar-rand1-human-agg-infl-mem.npz') # Access individual components total_runs = data['total_runs'] trainset_mask = data['trainset_mask'] memorization = data['memorization'] Notes Our results were aggregated over 1500 training runs. For more details or questions, feel free to reach out!