The power system is becoming increasingly complex due to the rise of distributed renewable energy sources and controllable loads such as batteries and electric vehicles. To effectively manage such complexity, a cellular approach emerges as a promising strategy, organising the system into cells that autonomously balance power locally while interacting with neighbouring cells. However, ensuring secure and reliable operation in such complex power systems demands comprehensive simulations of both the power system models and the necessary communication infrastructure between cells, along with their associated algorithms before testing in the field. Co-simulation approaches are positional to address such a problem. This work demonstrates utilising open-source tools to simulate cellular power grids effectively. We present a co-simulation framework that includes a multi-agent system simulator for cellular distributed power systems. The framework enables the development of heterogeneous scenarios with a wide range of energy system components and adaptors supported by the mosaik ecosystem. Moreover, it enables simultaneous peer-to-peer (between-cell) and hierarchical (within-cell) agent interaction with user-defined behaviour. We validate our solution using a scenario that is derived from the European configuration of the medium voltage distribution networks and perform computational experiments that confirm the high scalability of the proposed solution.