We simultaneously measure the static friction and the real area of contact between two solid bodies. These quantities are traditionally considered equivalent, and under static conditions both increase logarithmically in time, a phenomenon coined aging. Here we show that the frictional aging rate is determined by the combination of the aging rate of the real area of contact and two memory-erasure effects that occur when shear is changed (e.g., to measure static friction.) The application of a static shear load accelerates frictional aging while the aging rate of the real area of contact is unaffected. Moreover, a negative static shear-pulling instead of pushing-slows frictional aging, but similarly does not affect the aging of contacts. The origin of this shear effect on aging is geometrical. When shear load is increased, minute relative tilts between the two blocks prematurely erase interfacial memory prior to sliding, negating the effect of aging. Modifying the loading point of the interface eliminates these tilts and as a result frictional aging rate becomes insensitive to shear. We also identify a secondary memory-erasure effect that remains even when all tilts are eliminated and show that this effect can be leveraged to accelerate aging by cycling between two static shear loads.