This paper investigates decision quality in large choice sets across several choice architectures in three studies. In the first controlled experiment, we manipulate two features of a choice architecture—the response mode (for ranking alternatives) and presentation mode (for presenting alternatives). Our design objectively ranks all 16 choice options in each choice set and makes it possible to observe decision quality directly, independent of attitudes toward risk. We find joint presentation outperforms separate presentation and that choice response modes outperform “happiness ratings,” which outperform hypothetical monetary valuations. We also apply classic welfare criteria to assess the performance of the architectures. Our key finding is that low cognitive reflection subjects (as measured by the cognitive reflection test) perform better given a large choice set than given smaller sets collectively containing the same alternatives. This illustrates a basic tradeoff confronting choice architectures: for a fixed choice set, fewer options improve decision quality within that set but require architectures to elicit multiple responses, increasing opportunities for errors. One follow-up study demonstrates the robustness of the response mode result in a comparison using the tournament presentation mode. A second follow-up study reveals that the impact of incentivizing monetary valuations depends on cognitive reflection.