Despite the success of the WHO-led smallpox eradication programme a quarter of a century ago, there remains considerable fear that variola virus, or other related pathogenic poxviruses such as monkeypox, could re-emerge and spread disease in the human population. Even today, we are still mostly ignorant about why most poxvirus infections of vertebrate hosts show strict species specificity, or how zoonotic poxvirus infections occur when poxviruses occasionally leap into novel host species. Poxvirus tropism at the cellular level seems to be regulated by intracellular events downstream of virus binding and entry, rather than at the level of specific host receptors as is the case for many other viruses. This review summarizes our current understanding of poxvirus tropism and host range, and discusses the prospects of exploiting host-restricted poxvirus vectors for vaccines, gene therapy or tissue-targeted oncolytic viral therapies for the treatment of human cancers.