Ultrasonic industrial and medical imaging applications involve acquisition of large amount of volumetric data in real time. Therefore, data storage becomes critical in many current day applications which utilise ultrasound technology. Compressing the acquired data allows possessing minimal storage and also helps to rapidly transmit information to remote locations for expert analysis. The objective of this study is to design computationally efficient architectures for implementing discrete wavelet transform‐based ultrasonic three‐dimensional (3D) data compression algorithm on a reconfigurable ultrasonic system‐on‐chip (SoC) hardware platform. In this study, hardware and software architectures of the 3D ultrasonic compression algorithm are realised using Xilinx Zynq all programmable SoC. This study demonstrates that, compressing 33 MB of experimental ultrasonic 3D data into 0.42 MB (98.7% compression) requires only 84 ms for hardware architecture, and 1 min for software architecture, making both designs highly suitable for real‐time ultrasonic imaging applications. Furthermore, the 3D compression is implemented by using Open Computing Language (OpenCL) targeted on Nvidia GT 750M graphical processing unit. OpenCL implementation of ultrasonic 3D compression algorithm completes the execution in <1 sec. This approach provides improved computational performance as that of hardware architecture, and comparable flexibility as that of software implementation.