Abstract Bifactor Item Response Theory (IRT) models are the usual option for modeling composite constructs. However, in application, researchers typically must assume that all dimensions of person parameter space are orthogonal. This can result in absurd model interpretations. We propose a new bifactor model—the Completely Oblique Rasch Bifactor (CORB) model—which allows for estimation of correlations between all dimensions. We discuss relations of this model to other oblique bifactor models and study the conditions for its identification in the dichotomous case. We analytically prove that this model is identified in the case that (a) at least one item loads solely on the general factor and no items are shared between any pair of specific factors (we call this the G-structure), or (b) if no items load solely on the general factor, but at least one item is shared between every pair of the specific factors (the S-structure). Using simulated and real data, we show that this model outperforms the other partially oblique bifactor models in terms of model fit because it corresponds to the more realistic assumptions about construct structure. We also discuss possible difficulties in the interpretation of the CORB model’s parameters using, by analogy, the “explaining away” phenomenon from Bayesian reasoning.