Cloud Radio Access Network (C-RAN) has been proposed as a potential solution to reduce network power consumption, while providing acceptable quality of service (QoS). In this context, the conventional base station is separated into a Base Band Unit (BBU) and a Remote Radio Head (RRH). The BBUs are located in a cloud data center, whereas the RRHs are geographically distributed across multiple sites. To achieve statistical multiplexing gain, many RRHs may be clustered and associated with a single BBU. We formulate the RRH clustering as a multi-objective optimization problem, using the weighted-sum method and the $\epsilon$-constraint method. Our objectives are to minimize network power consumption and transmission delay. As these formulations result in non-linear problems, exhaustive search is used to obtain optimal solutions. Simulation results compare our solutions against the no-clustering solution, where each RRH is associated with a separate BBU, and the grand coalition solution, where all RRHs are associated with a single BBU. We further investigate the trade-off between our two crucial but conflicting objectives.