T cell receptors (TCRs) have emerged as a new class of immunological therapeutics. However, though antigen specificity is a hallmark of adaptive immunity, TCRs themselves do not possess the high specificity of monoclonal antibodies. Although a necessary function of T cell biology, the resulting cross-reactivity presents a significant challenge for TCR-based therapeutic development, as it creates the potential for off-target recognition and immune toxicity. Efforts to enhance TCR specificity by mimicking the antibody maturation process and enhancing affinity can inadvertently exacerbate TCR cross-reactivity. Here we demonstrate this concern by showing that even peptide-targeted mutations in the TCR can introduce new reactivities against peptides that bear similarity to the original target. To counteract this, we explored a novel structure-guided approach for enhancing TCR specificity independent of affinity. Tested with the MART-1-specific TCR DMF5, our approach had a small but discernible impact on cross-reactivity toward MART-1 homologs yet was able to eliminate DMF5 cross-recognition of more divergent, unrelated epitopes. Our study provides a proof of principle for the use of advanced structure-guided design techniques for improving TCR specificity, and it suggests new ways forward for enhancing TCRs for therapeutic use.