We present the current status of our project of developing a photon counting detector for medical imaging. An example motivation lays in producing a monitoring and dosimetry device for boron neutron capture therapy, currently not commercially available. Our approach combines in-house developed detectors based on cadmium telluride or thick silicon with readout chip technology developed for particle physics experiments at CERN. Here we describe the manufacturing process of our sensors as well as the processing steps for the assembly of first prototypes. The prototypes use currently the PSI46digV2.1-r readout chip. The accompanying readout electronics chain that was used for first measurements will also be discussed. Finally we present an advanced algorithm developed by us for image reconstruction using such photon counting detectors with focus on boron neutron capture therapy. This work is conducted within a consortium of Finnish research groups from Helsinki Institute of Physics, Aalto University, Lappeenranta-Lahti University of Technology LUT and Radiation and Nuclear Safety Authority (STUK) under the RADDESS program of Academy of Finland. Keywords: Solid state detectors, X-ray detectors, Gamma detectors, Neutron detectors, Instrumentation for hadron therapy, Medical-image reconstruction methods and algorithms.