The popularity of the Function-As-a-Service (FaaS) computing paradigm has exceeded the borders of Cloud data centers, aiming to bring the benefits of serverless computing at the edge of the network as well. Enjoying both the reduced latency of Edge and the resource richness of Cloud calls for suitable architectures and strategies, especially as regards policies for function offloading to make efficient use of the available resources. In this paper, we devise Quality-Of-Service-Aware offloading policies for serverless functions through deep reinforcement learning (DRL). The proposed approach learns how to optimize the utility generated over time and the incurred operational costs, keeping into account the service requirements of heterogeneous functions and users. Experiments on a FaaS prototype demonstrate the effectiveness of the approach, compared to a model-based competitor, at the price of increased computational demand due to DRL training.