In this extended abstract, we propose a theoretical framework for knowledge transfer between domains, e.g. from simulation to the real world. The framework is based on the concept of input-output abstraction, whose goal is to minimize the distance between domains while keeping enough information to solve the task. The value of this framework is twofold. First, it provides the theoretical background for justifying the empirical finding of prior work that intermediate representations are useful for action. Second, it suggests a practical algorithm for zero-shot simulation to reality transfer. We demonstrate this framework on two challenging tasks: drone racing and high-speed navigation in the wild. A video demonstrating the applications can be found at: https://youtu.be/uTWcC6IBsE4.