As a variation of ordinary digital signature scheme, a proxy signature scheme enables a proxy signer to sign messages on behalf of the original signer. Proxy multi-signature is an extension of the basic proxy signature primitive, and permits two or more entities to delegate their signing capabilities to the same other entity. Combining proxy multi-signature with identity based cryptography, in this paper, we define security notions for identity based proxy multi-signature schemes. We also construct a concrete identity based proxy multi-signature scheme which is provably secure in the random oracle model under the computational Diffie-Hellman assumption over pairing-friendly groups. Furthermore, the new scheme is very simple and efficient computationally. It has the property that the size of a proxy multi-signature is independent of the number of the original signers.