In passive infrared localization (PIL) humans are located based on their thermal radiation. Thus, an active tag is not required and privacy is guaranteed due to non-identifying sensors. However, in case of multi-target tracking, the non-identifying sensors result in missing associations between targets and measurements. As additionally the number of humans in the surveillance region is unknown and varying, conventional localization approaches like the JPDA or MHT filter cannot be applied since they require a fixed number of targets or are not tractable.