Surgical tool detection is a key component for analysing surgical workflow and operative activities. The power of deep learning approaches has been widely investigated for processing and recognising the content of laparoscopic images. However, proposed methods, so far, were trained and evaluated using data acquired from a single source. In this work, we evaluate the performance of a convolutional neural network (CNN) model to detect surgical tools in images obtained from different sources. The evaluation results show a drop in the model performance when the evaluation set and training set are not from the same source.