In this paper we present a novel high performance, low resource utilization and power efficient hardware architecture of an entropy coding scheme. The proposed architecture implements the Hierarchical Enumerative Coding algorithm (HENUC) on an embedded soft-processor based System-on-Chip, in which HENUC is an integral part of a wavelet based encoder oriented for locally stationary image source. Though HENUC has been implemented on an embedded DSP architecture before, the throughput was low. This paper proposes an optimized parallel architecture for HENUC, which is validated on a Terasic DE4-230 board containing Altera Stratix IV FPGA. Our implementation at 100MHz provides 5.7x speedup over Intel Xeon 8-core CPU and 12.3x speedup over TI DSP for 512 × 512 image while consuming less than 500 mw FPGA core power.