The prime goal of this research is to model the long-range dependency and volatility factors fitting in fractionally differenced ARMA (ARFIMA) and Gegenbauer ARMA processes (GARMA) in financial time series. This extends the efficiency in computing the exact maximum likelihood established by Sowell through conditional quasi maximum likelihood (QMLE) for ARFIMA and GARMA with conditional heteroscedastic errors. In particular, an extended algorithm together with corresponding asymptotic results of QMLE estimators are presented. The Monte Carlo simulation methods are used to study asymptotic properties and report the convergence rate for parameter estimates. Portmanteau test statistics are employed to check the model adequacy. As an application of this theory in the financial industry, a GARMA-GARCH model is fitted to daily returns of China Shanghai Composite stock index.