Noninvasive imaging of the reporter gene expression based on bioluminescence is playing an important role in the areas of cancer biology, cell biology, and gene therapy. The central problem for the bioluminescence tomography (BLT) we are developing is to reconstruct the underlying bioluminescent source distribution in a small animal using a modality fusion approach. To solve this inversion problem, a mathematical model of the mouse is built from a CT/micro-CT scan, which enables the assignment of optical parameters to various regions in the model. This optical geometrical model is used in the Monte Carlo simulation to calculate the flux distribution on the animal body surface, as a key part of the BLT process. The model development necessitates approximations in surface simplification, and so on. It leads to the model mismatches of different kinds. To overcome such discrepancies, instead of developing a mathematical model, segmented CT images are directly used in our simulation software. While the simulation code is executed, those images that are relevant are assessed according to the location of the propagating photon. Depending upon the segmentation rules including the pixel value range, appropriate optical parameters are selected for statistical sampling of the free path and weight of the photon. In this paper, we report luminescence experiments using a physical mouse phantom to evaluate this image-guided simulation procedure, which suggest both the feasibility and some advantages of this technique over the existing methods.