A novel nano-porous 3D architecture of N-doped carbon nanorods arrays grown on the surface of graphene has been prepared by carbonizing polyaniline/graphene oxide (PANI-GO) composite with PANI nanorod arrays on both sides of GO nanosheets. The obtained carbon materials are entirely composed of regularly grown carbon nanorods on graphene with height of about 100 nm and width about 30 nm, showing porous property due to the decomposition of PANI chains. The morphology of PANI grown on GO at the different growth stages was investigated to demonstrate the mechanism of the finally hierarchical architecture formation. Due to its large specific surface area and incorporation of the nitrogen groups into the carbon matrix, the obtained 3D carbon material enhances the ionic transport and the super-capacitance by synergetic effect of both double-layer and faradaic capacitances. This study provides a controllable approach to fabricate hierarchical carbon material based on conducting polymers and graphene oxide with promising applications in the high-rate electrode material of supercapacitors.