The activity-dependent rules that govern the wiring of GABAergic interneurons are not well understood. Chandelier cells (ChCs) are a type of GABAergic interneuron that control pyramidal cell output through axo-axonic synapses that target the axon initial segment. In vivo imaging of ChCs during development uncovered a narrow window (P12-P18) over which axons arborized and formed connections. We found that increases in the activity of either pyramidal cells or individual ChCs during this temporal window result in a reversible decrease in axo-axonic connections. Voltage imaging of GABAergic transmission at the axon initial segment (AIS) showed that axo-axonic synapses were depolarizing during this period. Identical manipulations of network activity in older mice (P40-P46), when ChC synapses are inhibitory, resulted instead in an increase in axo-axonic synapses. We propose that the direction of ChC synaptic plasticity follows homeostatic rules that depend on the polarity of axo-axonic synapses.