The paper provides a review of how to estimate a probability of failure from a small sample of data, and shows that the usual estimators of the parameters of the cumulative distribution function are biased, and can lead to unconservative estimations. Then, it explores different ways to make this estimation conservative: one is based on adding constraints when distributions are fitted; the second is based on the use of bootstrap methods. We explore the relationship between the chance that the estimate is conservative and the accuracy of the estimate. In particular, we study the case when we want to achieve a 95% chance to have conservative estimators. Finally, these methods are applied to the problem of a composite panel under thermal loading.