Using inductive cards as a model (Gardner 1959 Inductive Cards. Scientific American 200:160), I devised a lab for a course in statistics for graduate and upper level undergraduate university students. Students worked in groups of three or four. One person (Nature) devises a rule for placing cards in piles. The other students in the group work together to infer a rule for cards placed by Nature according to the unknown rule. On the first round cards are drawn from a shuffled deck. This is an observational study with an uncontrolled random component. On the second round (Selected cards) each rule moves to a different group, where students chose cards to present to Nature (an experimental study). On the third round a new group applies the strong inference (SI) method to a rule. The lab required students to list multiple working hypotheses at each step, list one or more “crucial test” cards, present them to Nature for placement, and disconfirm one or more hypotheses. The procedure is repeated until the rule is discovered. The number of cards to infer the rule is tallied by the instructor and distributed to students for their lab write-ups.

Nearly sixty years ago, in a publication with a growing rate of citation ever since, JR Platt presented “strong inference” (SI) as an accumulative method of inductive inference to produce much more rapid progress than others. The article offered persuasive testimony for the use of multiple working hypotheses combined with disconfirmation. It is often cited as an exemplar of scientific practice. However, the article provides no evidence of greater efficacy. Over a 34 year period a total 780 matched trials were completed in 56 labs in a university course in statistical science. The reduction from random (18.9 cards) to selected cards was 7.2 cards, compared to a further reduction of 0.3 cards from selected to SI. In 46% of the 780 trials, the number of cards to infer a rule was greater for strong inference than for a less rigid experimental method. Based on the evidence, strong inference added little additional strength beyond that of less rigidly structured experiments.

The data were intially stored as ascii (.txt) files (1989 to 2003). This data was moved to excel files in 2023. Data from 2004 through 2022 were stored as excel files.