Abstract The definition of a confidence interval is generalized so that problems such as constructing exact confidence regions for the difference in two normal means can be tackled without the assumption of equal variances. Under certain conditions, the extended definition is shown to preserve a repeated sampling property that a practitioner expects from exact confidence intervals. The proposed procedure is also applied to the problem of constructing confidence intervals for the difference in two exponential means and for variance components in mixed models. A repeated sampling property of generalized p values is also given. With this characterization one can carry out fixed level tests of parameters of continuous distributions on the basis of generalized p values. Finally, Pratt's paradox is revisited, and a procedure that resolves the paradox is given.