No two physical objects are exactly the same, even when manufactured with a nominally identical process. For example, two sheets of paper that are indistinguishable with the naked eye, still differ considerably in their nanoscale fiber structures. Although manufacturing variability is usually undesired, the associated ability to uniquely identify a physical object, which is constrained to an integrated circuit (IC) in this thesis, can be leveraged for security purposes. To facilitate the registration of unique features, a so-called physically unclonable function (PUF) can be implemented on the IC. A PUF circuit is designed to be sensitive to process variations, i.e., challenged with a binary input, it provides a binary, device-unique response. This building block can hence be understood as the silicon equivalent of human biometrics. PUFs can augment the security architecture of an ever-increasing number of electronic devices that access our personal data and/or represent our identities. This includes but is not limited to smartphones, credit cards, access badges, the sensors and actuators of automated home, and medical implants. PUFs usually need to team up with other building blocks, e.g., true random number generators (TRNGs), cryptographic algorithms, error-correcting codes, nonvolatile memory (NVM), etc. We analyze the security of such multi-component systems in a format that allows for comparisons among proposals that have similar or identical objectives. Numerous newly revealed flaws and attacks are presented throughout this thesis. On the bright side, the lessons learned can help improve the quality of future PUF-based systems. This work was partially funded by the European Union's Horizon 2020 research and innovation programme under grant agreement number 644052 (HECTOR).