The high performance requirements at the European Spallation Source have been driving the technological advances on the neutron detector front. Now more than ever is it important to optimize the design of detectors and instruments, to fully exploit the ESS source brilliance. Most of the simulation tools the neutron scattering community has at their disposal target the instrument optimization until the sample position, with little focus on detectors. The ESS Detector Group has extended the capabilities of existing detector simulation tools to bridge this gap. An extensive software framework has been developed, enabling efficient and collaborative developments of required simulations and analyses -- based on the use of the Geant4 Monte Carlo toolkit, but with extended physics capabilities where relevant (like for Bragg diffraction of thermal neutrons in crystals). Furthermore, the MCPL (Monte Carlo Particle Lists) particle data exchange file format, currently supported for the primary Monte Carlo tools of the community (McStas, Geant4 and MCNP), facilitates the integration of detector simulations with existing simulations of instruments using these software packages. These means offer a powerful set of tools to tailor the detector and instrument design to the instrument application.