We examine whether hedging effectiveness is affected by asymmetry in the return distribution by applying tail specific metrics to compare the hedging effectiveness of short and long hedgers using crude oil futures contracts. The metrics used include Lower Partial Moments (LPM), Value at Risk (VaR) and Conditional Value at Risk (CVAR). Comparisons are applied to a number of hedging strategies including OLS and both Symmetric and Asymmetric GARCH models. Our findings show that asymmetry reduces in-sample hedging performance and that there are significant differences in hedging performance between short and long hedgers. Thus, tail specific performance metrics should be applied in evaluating hedging effectiveness. We also find that the Ordinary Least Squares (OLS) model provides consistently good performance across different measures of hedging effectiveness and estimation methods irrespective of the characteristics of the underlying distribution.