Visual cortex functionality in the blind has been shown to shift away from sensory networks toward task-positive networks that are involved in top-down modulation. However, how such modulation is shaped by experience and reflected behaviorally remains unclear. This study evaluates the visual cortex activity and functional connectivity among congenitally blind, acquired blind, and sighted subjects using blood-oxygenation-level-dependent functional MRI during sensory substitution tasks and at rest. We found that primary visual cortex activity due to active interpretation not only depends on the blindness duration, but also negatively associates with behavioral reaction time. In addition, alterations in visual and task-positive functional connectivity progress over the duration of blindness. In summary, this work suggests that functional plasticity in the primary visual cortex can be reshaped in the blind over time, even in the adult stage. Furthermore, the degree of top-down activity in the primary visual cortex may reflect the speed of performance during sensory substitution.