Self-improving control systems may belong to either of two categories, according to whether or not they embody an explicit model of the part of their environment with which they interact. The two forms of operation are discussed and compared, and it is shown that the two may be mathematically equivalent. The treatment also gives theoretical justification for a particular mode of operation for nonmodel-forming controllers.