<p>This paper presents a visual detection of fingertips by using a classification technique based on the bag-of-words method. In this work, the fingertips are specifically of people who are are holding a therapy ball, as it is intended to be used in a hand rehabilitation project. Speeded Up Robust Features (SURF) descriptors are used to generate feature vectors and then the bag-of-feature model is constructed by K-mean clustering which reduces the number of features. Finally, a Support Vector Machine (SVM) is trained to produce a classifier that distinguishes whether the feature vector belongs to a fingertip or not. A total of 4200 images, 2100 fingertip images and 2100 non-fingertip images, were used in the experiment. Our results show that the success rates for the fingertip detection are higher than 94% which demonstrates that the proposed method produces a promising result for fingertip detection for therapy-ball-holding hands.</p>