This paper is about a set-based computing method for solving a general class of two-player zero-sum Stackelberg differential games. We assume that the game is modeled by a set of coupled nonlinear differential equations, which can be influenced by the control inputs of the players. Here, each of the players has to satisfy their respective state and control constraints or loses the game. The main contribution is a backward-forward reachable set splitting scheme, which can be used to derive numerically tractable conservative approximations of such two player games. In detail, we introduce a novel class of differential inequalities that can be used to find convex outer approximations of these backward and forward reachable sets. This approach is worked out in detail for ellipsoidal set parameterizations. Our numerical examples illustrate not only the effectiveness of the approach, but also the subtle differences between standard robust optimal control problems and more general constrained two-player zero-sum Stackelberg differential games.