Visual backward masking is a commonly used technique in vision research and psychology. There are two distinct types of masking. Either masking is strongest for a simultaneous presentation of the target and the mask (A-type masking) or masking is strongest when the mask trails the target (B-type masking). To account for the two types of masking, a variety of explanations have been put forward that often rely on low-level features such as the target-mask energy ratio. However, recent studies have demonstrated that the global spatial layout of the mask is an equally important factor. Here, we investigated both factors jointly. Our findings show that both factors strongly interact with each other and that neither one alone can explain the results. This finding indicates that choosing a mask should not be taken lightly when masking is used as a tool to investigate properties of perception or cognition.