Video is the most powerful and, at the same time, the most complex of all media used for conveying information. Hence, representing video information to enable effective and efficient retrieval is an interesting problem. In this paper, we first discuss the architecture of our video retrieval system.Then we discuss a set of metrics for validating this system through measurements of it's ‘effectiveness’ and ‘efficiency’. Effectiveness is characterized through the metrics of Recall and Precision for ‘exact match’ queries, and through F-Effectiveness for ‘approximate match’ queries. Efficiency is characterised through a metric called EM. We also show how to fine-tune a system using these metrics. Considering a video database of visuals of a cricket match, the details of the work are discussed.