Surfactant adsorption in a three-phase system and its influence on wetting properties are relevant in various applications. Here, we report a hitherto not observed phenomenon, namely the retraction of an aqueous drop on hydrophilic solid substrates (which we refer to as 'autophobing') in ambient oil containing water-insoluble fatty acids, caused by the deposition of these fatty acids from the ambient oil onto the solid substrate. AFM measurements confirm that the surfactant is deposited on the solid by the moving contact line. This leads to a more hydrophobic substrate, the retraction of the contact line and a concomitant increase in the contact angle. The deposition process is enabled by the formation of a reaction product between deprotonated fatty acids and Ca(2+) ions at the oil/water interface. We investigate how the transition to a new equilibrium depends on the concentrations of the fatty acids, the aqueous solute, the chain lengths of the fatty acid, and the types of alkane solvent and silica or mica substrates. This phenomenon is observed on both substrates and for all explored combinations of fatty acids and solvents and thus appears to be generic. In order to capture the evolution of the contact angle, we develop a theoretical model in which the rate of adsorption at the oil-water interface governs the overall kinetics of autophobing, and transfer to the solid is determined by a mass flux balance (similar to a Langmuir Blodgett transfer).