The immune system is a fundamental part of the tumor microenvironment. In particular, cytotoxic lymphocytes, such as cytolytic T cells and natural killer cells, control tumor growth and disease progression by interacting and eliminating tumor cells. The actin cytoskeleton of cytotoxic lymphocytes engaged in an immunological synapse has received considerable research attention. It has been recognized as a central mediator of the formation and maturation of the immunological synapse, and its signaling and cytolytic activities. In comparison, fewer studies have explored the organization and function of actin filaments on the target cancer cell side of the immunological synapse. However, there is growing evidence that the actin cytoskeleton of cancer cells also undergoes extensive remodeling upon cytotoxic lymphocyte attack, and that such remodeling can alter physical and functional interactions at the immunological synapse. In this article, we review the current knowledge of actin organization and functions at both sides of the immunological synapse between cytotoxic lymphocytes and cancer cells, with particular focus on synapse formation, signaling and cytolytic activity, and immune evasion.