This work conceives techniques for the design of hybrid precoders/combiners for optimal bit allocation in frequency selective millimeter wave (mmWave) multiple-input multiple-output (MIMO) orthogonal frequency division multiplexing (OFDM) systems, toward transmission rate maximization. Initially, the optimal fully digital ideal precoder/ combiner design is derived together with a closed-form expression for the optimal bit allocation in the above system. This is followed by the development of a framework for optimal transceiver design and bit allocation in a practical mmWave MIMO-OFDM implementation with a hybrid architecture. It is demonstrated that the pertinent problem can be formulated as a multiple measurement vector (MMV)-based sparse signal recovery problem for joint design of the RF and baseband components across all the subcarriers, and an explicit algorithm is derived to solve this using the simultaneous orthogonal matching pursuit (SOMP). To overcome the shortcomings of the SOMP-based greedy approach, an MMV sparse Bayesian learning (MSBL)-based state-of-the-art algorithm is subsequently developed, which is seen to lead to improved performance due to the superior sparse recovery properties of the Bayesian learning framework. Simulation results verify the efficacy of the proposed designs and also demonstrate that the performance of the hybrid transceiver is close to that of its fully-digital counterpart.