Many cryptographers believe that the only way to have confidence in the security of a cryptographic protocol is to have a mathematically rigorous proof that the protocol meets its stated goals under certain assumptions. However, it is often difficult to assess what such proofs really mean in practice especially if the proof is non-tight, the underlying assumptions are contrived, or the security definition is in the single-user setting. We will present some examples that illustrate this difficulty and highlight the important role that old-fashioned cryptanalysis and sound engineering practices continue to play in establishing and maintaining confidence in the security of a cryptographic protocol.