The design of secure systems requires the use of security analysis techniques. Security objectives have to be considered during the early stages of system development and design; an executable model will give the designer the advantage of exploring the vulnerabilities early, and therefore enhancing the system security. In this work we create an executable model of a smart card system using SystemC with the Transaction Level Modelling (TLM) extensions. The model includes the security protocols and transactions. The model is used to compare a number of authentication mechanisms with different probabilities of failure. In addition, a number of probable attacks, including theft of a private key and denial of service were modelled to examine the vulnerabilities. The executable model shows that security protocols and transactions can be effectively simulated in order to design improvements to withstand different types of security attacks.