It is becoming increasingly clear that inflammation influences prostate cancer (PCa) development and that immune cells are among the primary drivers of this effect. This information has launched numerous clinical trials testing immunotherapy drugs in PCa patients. The results of these studies are promising but have yet to generate a complete response. Importantly, the precise immune profile that determines clinical outcome remains unresolved. Individual immune cell types are divided into various functional subsets whose effects on tumor development may differ depending on their particular phenotype and functional status, which is often shaped by the tumor microenvironment. Thus, this review aims to examine the current knowledge regarding the role of inflammation and specific immune cell types in mediating PCa progression to assist in directing and optimizing immunotherapy targets, regimens, and responses and to uncover areas in which further research is needed. Finally, a summary of ongoing immunotherapy clinical trials in PCa is provided.