In this paper, the relationship between information and reasoning is investigated and a parallel reasoning method is proposed based on information theory, in particular the principle of minimum cross entropy. Some technical issues, such as multiple uncertain evidence, complicated constraints, small directed cycles and decomposition of underlying networks, are discussed. Some simple examples are also given to compare the method proposed here with other methods.