Abstract In most sensory studies the products are evaluated one after another. There is concern that the perception of assessors might be influenced by carryover effects. Therefore it might be useful to use a design which is balanced for carryover effects and to not randomize the order in which the products are tasted. However, this leads to another problem: if one uses a systematic design, then the usual randomization argument for independent identically distributed errors no longer holds. The paper utilizes simulation to demonstrate that randomization validates an analysis. It also demonstrates that carryover effects can destroy this validity. However, randomization is possible, even if the presence of carrover effects is suspected. There are two randomization procedures which leave balance for carryover effects intact, but validate simple analyses: one justifies the simple block model, the other the row-column model. The simulations demonstrate the influence of carryover effects if one of these two randomization procedures is done. Finally, the performance of an analysis which corrects for carryover is explored.