Unlike electro-acoustic sound sources, musical instruments have a time-varying, dynamic directivity, due to the note-dependent radiation behavior of the instrument and due to the expressive movements that musicians perform with their instrument. While previous studies have generally examined the directivity of the static, unmoved instrument for specific notes played, we show the individual and combined contributions of these two factors to a temporal modulation of the radiation behavior, based on motion tracking of typical movement patterns for all instruments of a classical symphony orchestra and on the directivity measured for all partials over the entire pitch range of these instruments. The effect of this modulation, which is manifested by changes in timbre and room acoustic excitation, was determined by spectral variations in the free field and under reverberant conditions, as well as by a modulation of room acoustic parameters. Our results show that these effects are well above the just noticeable differences for all musical instruments and all perceptual variables considered. While the effect of motion dominates for brass instruments, string and woodwind instruments exhibit large note-related differences, which should be taken into account in virtual acoustic realities if an auditory liveliness comparable to physical reality is to be achieved.