Since the discovery of the paradoxes of self-referentiality or self-reference respectively logicians and mathematicians tried to avoid self-reference when constructing formal systems. Yet “real” complex systems like the mind are characterized by self-reference and can accordingly only be modeled by formal systems that are also basically self-referential. In this article I show that and how self-referential computer programs, understood as algorithmic formal systems, are not only possible but also since some time quite common in special branches of computer science. Examples for this argument are neural networks and so-called hybrid systems, i.e. combination of different sub systems. The hybrid system SOCAIN, a combination of a cellular automaton, a neural network and a genetic algorithm is an example for the fruitfulness of using self-reference in a systematic way. In particular, such systems consist of mutually dependent sub systems, i.e. form no static hierarchy.