Context. The fate of a massive star during the latest stages of its evolution is highly dependent on its mass-loss rate and geometry and therefore knowing the geometry of the circumstellar material close to the star and its surroundings is crucial. Aims. We aim to provide insight into the nature (i.e. geometry, rates) of mass-loss episodes, and in particular, the connection between the observed asymmetries due to the mass lost in a fast wind or during a previous, prodigious mass-losing phase. In this context, yellow hypergiants offer a good opportunity to study mass-loss events. Methods. We analysed a large set of optical and near-infrared data in spectroscopic and photometric, spectropolarimetric, and interferometric (GRAVITY/VLTI) modes, towards the yellow hypergiant IRAS 17163−3907. We used X-shooter optical observations to determine the spectral type of this yellow hypergiant and we present the first model-independent, reconstructed images of IRAS 17163−3907 at these wavelengths tracing milli-arcsecond scales. Lastly, we applied a 2D radiative transfer model to fit the dereddened photometry and the radial profiles of published diffraction-limited VISIR images at 8.59 μm, 11.85 μm, and 12.81 μm simultaneously, adopting a revised distance determination using Gaia Data Release 2 measurements. Results. We constrain the spectral type of IRAS 17163−3907 to be slightly earlier than A6Ia (Teff ∼ 8500 K). The interferometric observables around the 2 μm window towards IRAS 17163−3907 show that the Brγ emission appears to be more extended and asymmetric than the Na I and the continuum emission. Interestingly, the spectrum of IRAS 17163−3907 around 2 μm shows Mg II emission that is not previously seen in other objects of its class. In addition, Brγ shows variability in a time interval of four months that is not seen towards Na I. Lastly, in addition to the two known shells surrounding IRAS 17163−3907, we report on the existence of a third hot inner shell with a maximum dynamical age of only 30 yr. Conclusions. The 2 μm continuum originates directly from the star and not from hot dust surrounding the stellar object. The observed spectroscopic variability of Brγ could be a result of variability in the mass-loss rate. The interpretation of the presence of Na I emission at closer distances to the star compared to Brγ has been a challenge in various studies. To address this, we examine several scenarios. We argue that the presence of a pseudo-photosphere, which was traditionally considered to be the prominent explanation, is not needed and that it is rather an optical depth effect. The three observed distinct mass-loss episodes are characterised by different mass-loss rates and can inform theories of mass-loss mechanisms, which is a topic still under debate both in theory and observations. We discuss these in the context of photospheric pulsations and wind bi-stability mechanisms.