We show that combining ideas from the fields of quantum invariants and optimal control can be used to design optimal quantum control solutions without explicit reference to quantum states. The states are specified only implicitly in terms of operators to which they are eigenstates. The scaling in numerical effort of the resultant approach is not given by the typically exponentially growing effort required for the specification of a time-evolved quantum state, but it is given by the effort required for the specification of a time-evolved operator. For certain Hamiltonians, this effort can be polynomial in the system size. We describe how control problems for state preparation and the realization of propagators can be formulated in this approach, and we provide explicit control solutions for a spin chain with an extended Ising Hamiltonian. The states considered for state-preparation protocols include eigenstates of Hamiltonians with more than pairwise interactions and these Hamiltonians are also used for the definition of target propagators. The cost of describing suitable time-evolving operators grows only quadratically with the system size, allowing us to construct explicit control solutions for up to 50 spins. While subexponential scaling is obtained only in special cases, we provide several examples that demonstrate favorable scaling beyond the extended Ising model. Published by the American Physical Society 2024