Loss of backup tapes containing personal information (PI) is a potential breach of privacy and encryption is the typical way to prevent the breach. This paper considers an attack scenario where an adversary who encrypts the PI for backup purpose tries to hide the plain PI in a valid-looking ciphertext without being detected. We show that the standard security notion IND-CCA2 does not capture such a scenario. For example, the Cramer-Shoup scheme is vulnerable to such an attack. To capture such a scenario, we define a new notion of “ciphertext-auditability” as a new property of public key encryption schemes (PKESs). It requires that, given a public key and a ciphertext, anyone should be able to verify whether the ciphertext was actually generated using the public key. Also, it requires that, given a public key and a plaintext, no adversary should be able to generate a valid-looking ciphertext so that the verification passes, but nevertheless the plaintext can be recovered from the ciphertext without the corresponding secret key. We propose a general construction of such PKESs based on standard cryptographic primitives in the random oracle model.