The impact of fingers on numerical cognition has received a great deal of attention recently. One sub-set of these studies focus on the relation between finger gnosis (also called finger sense or finger gnosia), the ability to identify and individuate fingers, and mathematical development. Studies in this subdomain have reported mixed findings so far. While some studies reported that finger gnosis correlates with or predicts mathematics abilities in younger children, others failed to replicate these results. The current study explores the relationship between finger gnosis and two arithmetic operations—addition and subtraction. Twenty-four second to third graders participated in this fMRI study. Finger sense scores were negatively correlated with brain activation measured during both addition and subtraction. Three clusters, in the left fusiform, and left and right precuneus were found to negatively correlate with finger gnosis both during addition and subtraction. Activation in a cluster in the left inferior parietal lobule (IPL) was found to negatively correlate with finger gnosis only for addition, even though this cluster was active both during addition and subtraction. These results suggest that the arithmetic fact retrieval may be linked to finger gnosis at the neural level, both for addition and subtraction, even when behavioral correlations are not observed. However, the nature of this link may be different for addition compared to subtraction, given that left IPL activation correlated with finger gnosis only for addition. Together the results reported appear to support the hypothesis that fingers provide a scaffold for arithmetic competency for both arithmetic operations.