Resource allocation and management in Cloud Computing is a very complex task. This is mainly due to the scale of the cloud and the number of services deployed in it. Since cloud users and service providers are given access to supercomputerlevel resources, their effect over the cloud's overall performance is greater than ever. This raises multiple research questions related to the management and performance of cloud computing systems in light of the end-users selfishness. In this work we specifically study the overall performance when selfish service providers may split work between the (shared) cloud and private resources. The size of modern data center and the number of service housed in it calls for fully distributed management solutions. We propose task assignment policies that are specifically adequate for large-scale distributed systems, and show that they provide new capabilities in improving system performance. In particular, we develop new resource allocation algorithms that converge to a working point that balances the end-user experience with the operational costs of leasing resources from the cloud provider.