Large Bayesian VARs are now widely used in empirical macroeconomics. One popular shrinkage prior in this setting is the natural conjugate prior as it facilitates posterior simulation and leads to a range of useful analytical results. This is, however, at the expense of modeling flexibility, as it rules out cross‐variable shrinkage, that is, shrinking coefficients on lags of other variables more aggressively than those on own lags. We develop a prior that has the best of both worlds: it can accommodate cross‐variable shrinkage, while maintaining many useful analytical results, such as a closed‐form expression of the marginal likelihood. This new prior also leads to fast posterior simulation—for a BVAR with 100 variables and 4 lags, obtaining 10,000 posterior draws takes less than half a minute on a standard desktop. We demonstrate the usefulness of the new prior via a structural analysis using a 15‐variable VAR with sign restrictions to identify 5 structural shocks.