In real world Service Oriented Software Systems, changes to web services infrastructure are not an exception but a rule. Timely detection and a fast reaction to handle these changes are important to reduce the impact. This paper proposes an agent based framework to manage the situation when third party web services become unavailable. It uses the concept of mobile agents to monitor web services on the provider side. The framework is implemented using Advanced JAVA and JADE environment. Experimental analysis shows that the proposed solution is both effective and efficient. In future, the framework will be extended to include monitoring of other Quality of Service properties.