This paper presents a wearable activity sensor system and a systematic activity classification scheme for the classification of human daily physical activities. The wearable activity sensor system, consisting of two activity sensor modules worn on users' dominant hand wrists and ankles, is used for collecting activity acceleration signals. The proposed activity classification scheme, including static/dynamic activity analysis, posture recognition, exercise classification, and ambulation classification, is capable of classifying time-series activity acceleration signals. The collected acceleration signals are classify into two categories by means of static/dynamic activity analysis. Posture recognition is applied for partitioning static signals into sitting and standing. Exercise classification and ambulation classification algorithms were used to classify dynamic activity signals. Our experimental results have successfully validated the effectiveness of the proposed wearable sensor system and the scheme of activity classification algorithms with an overall classification accuracy of 96% for seven types of daily activities.