Methods to calculate bias‐adjusted estimates for meta‐analyses are becoming more popular. The objective of this paper is to use the structural approach to bias and causal diagrams to show that (i) the current use of the bias‐adjusted estimating tools may sometimes introduce bias rather than reduce it and (ii) the Cochrane collaboration risk of bias tool, which was designed for randomized studies, is also applicable to non‐randomized studies with only minimal changes. Causal diagrams are used to illustrate each of the items in the current risk of bias tool and how they apply to both randomized and non‐randomized studies. With the exception of confounding by indication, the structure of all potential biases present in non‐randomized studies may also be present in randomized studies. In addition, causal diagrams demonstrate important limitations to the methods currently being developed to provide bias‐adjusted estimates of individual studies in meta‐analyses. Finally, causal diagrams can be helpful in deciding when it is appropriate to combine studies in a meta‐analysis of non‐randomized studies even though the studies may use different adjustment sets. Copyright © 2012 John Wiley & Sons, Ltd.