In this article, we analyse the potential of Large Language Models (LLMs) for social simulation by assessing their ability to: (a) make decisions aligned with explicit preferences; (b) adhere to principles of rationality; and (c) refine their beliefs to anticipate the actions of other agents. Through game-theoretic experiments, our results show that certain models, such as GPT-4.5 and Mistral-Small, exhibit consistent behaviours in simple contexts but struggle with more complex scenarios requiring anticipation of other agents' behaviour. Our study outlines research directions to overcome the current limitations of LLMs.