Abstract Site fidelity—the tendency to return to previously visited locations—is widespread across taxa. Returns may be driven by several mechanisms, including memory, habitat selection, or chance; however, pattern‐based definitions group different generating mechanisms under the same label of ‘site fidelity’, often assuming memory as the main driver. We propose an operational definition of site fidelity as patterns of return that deviate from a null expectation derived from a memory‐free movement model. First, using agent‐based simulations, we show that without memory, intrinsic movement characteristics and extrinsic landscape characteristics are key determinants of return patterns and that even random movements may generate substantial probabilities of return. Second, we illustrate how to implement our framework empirically to establish ecologically meaningful, system‐specific null expectations for site fidelity. Our approach provides a conceptual and operational framework to test hypotheses on site fidelity across systems and scales.