Replication is a big concern in contemporary science. At its core, replicable research can be repeated in different contexts, possibly using different tools, to obtain similar results. In geography, a recent replication study found a huge discrepancy: twice the number of significant "clusters" in local Moran's I statistics were detected in one software package relative to other packages. We show that this replication failure is a conceptual, not technical, issue: different "null hypotheses" used in local tests are not directly comparable. We show the conceptual issue, demonstrate it empirically, and then reflect on its implications for calls for “weak” replication in spatial science.