This folder contains anonymized data from the experiment described in the paper "Wrong Answers for Wrong Reasons: The Risks of Ad Hoc Instruments". questions.md contains the 14 questions used as pre/post-test Participants are identified using numeric identifiers (from 1 to 40). conditions.csv contains the mapping between participants identifiers and their condition {test}_{assessment_type}_anon.csv contains data for a given test (pretest or posttest) using a certain assessment type. Two assessment types are used: mc (multiple choice as assessed by Moodle: 0 means wrong, 1 means correct, - means missing) revised (human revision after reading the explanation: the judgment is either a 0 or a 1 following what presented in the paper) Statistics are computed with Python and SciPy. You can execute the script by running: python3 main.py and you should get this output: === Analyzing answers using mc as assessment type === Results on pretest - Condition Text: N: 21, mean ± std: 9.81 ± 3.03 Results on pretest - Condition Graphic: N: 19, mean ± std: 9.68 ± 2.11 Mann Whitney U on pretest scores: U=184.0, p=0.3406 Results on posttest - Condition Text: N: 21, mean ± std: 11.19 ± 2.75 Results on posttest - Condition Graphic: N: 19, mean ± std: 10.74 ± 2.23 Mann Whitney U on posttest scores: U=164.0, p=0.1681 === Analyzing answers using revised as assessment type === Results on pretest - Condition Text: N: 20, mean ± std: 9.15 ± 3.10 Results on pretest - Condition Graphic: N: 19, mean ± std: 7.89 ± 3.16 Mann Whitney U on pretest scores: U=150.5, p=0.1344 Results on posttest - Condition Text: N: 20, mean ± std: 11.15 ± 2.39 Results on posttest - Condition Graphic: N: 19, mean ± std: 9.42 ± 3.11 Mann Whitney U on posttest scores: U=123.0, p=0.0299 You might need to install scipy (e.g., depending on your setup, pip install scipy).