An autonomous agent is considered intelligent if it is capable to improve its performance through time, learning by experience. The problem of learning to improve performance from experience has been formalized in the field of Reinforcement Learning. The main difficulty found in this field is that the number of experiences required for learning to take place could be very large depending on the environment structure. If the environment has no structure at all the learning cannot be completed until all actions have been executed at least once in each possible state. Then, the application of reinforcement learning techniques is only fea-sible when the environment presents enough regularity to allow some kind of generalization. Many reinforcement learning techniques have been proposed using different forms of generalization. Despite the diversity of strategies proposed, usual generalization techniques do not take ad-vantage of all the opportunities of generalization. A new algorithm was proposed that exploits a type of regularity that is denoted as categorizability. Categorizability means that from all the relevant features that must be taken into account to decide the best action in any situation, only a few of them are actually relevant in each particular situation. In this paper the categorization and learning capabilities of the algorithm are evaluated using a problem that satisfies to a good extent the categorization property. The categorization achieved is analyzed in detail and illustrated with examples. The algorithm learning performance is compared with those of other learning algorithms. Some improvements of the original algorithm are introduced.

This work was supported by the project 'Sistema reconfigurable para la navegación basada en visión de robots caminantes y rodantes en entornos naturales.' (00).

Jornada de Recerca en Automàtica, Visió i Robòtica (AVR), 2004, Barcelona (España)

Peer Reviewed