Auditory perception involves cues in the monaural auditory pathways, as well as binaural cues based on interaural differences. So far, auditory models have often focused on either monaural or binaural experiments in isolation. Although binaural models typically build upon stages of (existing) monaural models, only a few attempts have been made to extend a monaural model by a binaural stage using a unified decision stage for monaural and binaural cues. A typical prototype of binaural processing has been the classical equalization-cancelation mechanism, which either involves signal-adaptive delays and provides a single channel output, or can be implemented with tapped delays providing a high-dimensional multichannel output. This contribution extends the (monaural) generalized envelope power spectrum model by a non-adaptive binaural stage with only a few, fixed output channels. The binaural stage resembles features of physiologically motivated hemispheric binaural processing, as simplified signal-processing stages, yielding a 5-channel monaural and binaural matrix feature “decoder” (BMFD). The back end of the existing monaural model is applied to the BMFD output and calculates short-time envelope power and power features. The resulting model accounts for several published psychoacoustic and speech-intelligibility experiments and achieves a prediction performance comparable to existing state-of-the-art models with more complex binaural processing.