QoS has been one of the major challenges in Web Services area. Though negotiated in the contract, service quality usually can not be guaranteed by providers. Therefore, the service composer is obligated to detect real quality status of component services. Local monitoring can not fulfill this task. We propose a Probe-based architecture to address this problem. By running light weighted test cases, the Probe can collect accurate quality data to support runtime service composition and composition re-planning.