{"references": ["Y.S. Abu-Mostafa, Learning from hints in neural networks, Journal of\nComplexity (1990) 6:192.", "B. Pandey and R.B. Mishra, Knowledge and intelligent computing\nsystem in medicine, Computers in Biology and Medicine, 39(3):215-230,\n2009.", "E. Barnard and D. Casasent, Invariance and neural networks, IEEE\nTransactions on Neural Networks, 2 :498-508, 1991.", "Y. Bengio, Neural Networks for Speech and Sequence Recognition,\nInternational Thompson Computer Press, 1996.", "C.M. Bishop, Neural Networks for Pattern Recognition, Oxford\nUniversity Press, 1995.", "K. David, D. Haussler, G. Martin Reese and H. Frank Eeckman, A\ngeneralized hidden Markov model for the recognition of human genes in\nDNA, Procedings of the Fourth International Conference on Intelligent\nSystems for Molecular Biology, AAAI Press, 1996.", "Richard O. Duda, Peter E. Hart, and David G. Stork. 2000. Pattern\nClassification (2nd Edition). Wiley-Interscience.", "J.L. Elman and D. Zipser, Learning the hidden structure of speech,\nJournal of the Acoustical Society of America, 83:1615-1626, 1988.", "L. Fu, Mapping rule-based systems into neural architecture, Knowledge\nBased Systems, 3(1):48-56, 1990.\n[10] Y. Fukuoka and H. Matsuki, A Modified Back-propagation Method to\nAvoid Local Minima, Neural Networks, 11:1059-1072, 1998.\n[11] M.J.F. Gales, Maximum likelihood linear transformations for Hidden\nMarkov Model-based speech recognition, Computer Speech Language,\n12:75-98, 1998.\n[12] A.S. Weigend and N.A. Gershenfeld, The Future of Time Series,\nLearning and Understanding, Addison-Wesley, Reading, MA, 1-17, 1993.\n[13] C.L. Giles, D. Chen, C.B. Miller, H.H. Chen, G.Z. Sun and Y.C.\nLee, Second-order recurrent neural networks for grammatical inference,\n1991 IEEE INNS International Joint Conference on Neural Networks,\nPiscataway, NJ, IEEE Press. Reprinted in Artificial Neural Networks,\neds. E. Sanchez-Sinencia, C. Lau, IEEE Press, 273-281, 1992.\n[14] Y. Hayashi and A. Imura, Fuzzy neural expert systems with automated\nextraction of fuzzy if-then rules from a trained neural network,\nProcedings of First IEEE Conference on Fuzzy Systems (1990) 489-494.\n[15] J. Hertz, A. Krogh and R.G. Palmer, Introduction to the Theory of Neural\nComputation. Addison-Wesley Publishing Company, 1991.\n[16] J.J. Hopfield, Neural networks and physical systems with emergent\ncollective computational facilities, Proceedings of the National Academy\nof Sciences of the USA, 79:2554\u20132558, 1982.\n[17] K. Hornik, M. Stinchcombe and H. White, Multilayer feedforward\nnetworks are universal approximators, Neural Networks, 2:359-366,\n1989.\n[18] J.F. Kolen and S.C. Kremer, (ed.), A Field Guide to Dynamical Recurrent\nNetworks, IEEE Press, Piscataway, NJ, 2001.\n[19] S. Lawrence, C. L. Giles and A.C. Tsoi, Symbolic conversion,\ngrammatical inference and rule extraction of foreign exchange rate\nprediction, Decision Technologies for Financial Engineering: Procedings\nof the Fourth International Conference on Neural Networks in the\nCapital Markets, World Scientific, Singapore (1998) 333-345.\n[20] K. Marakami and H Taguchi, Gesture recognition using recurrent neural\nnetworks, Procedings of the SIGCHI conference on Human factors in\ncomputing systems, 237-242, 1991.\n[21] L. R. Rabiner, A Tutorial on Hidden Markov Models and Selected\nApplications in Speech Recognition , Proceedings of the IEEE,\n77(2):257-285, 1989.\n[22] R.D. Reed and J. Robert Mark, Neural Smithing: Supervised Learning\nin Feedforward Artificial Neural Networks, The MIT Press, 1999.\n[23] A.J Robinson, An application of recurrent neural nets to phone\nprobability estimation, IEEE transactions on Neural Networks,\n5(2):298-305, 1994.\n[24] D.E. Rumelhart, J.L. McCLelland and the PDP Research Group, Parallel\nDistributed Processing: Explorations in the Microstructure of Cognition,\nFoundations, MIT Press, Cambridge, MA, 1, 1986.\n[25] G.G. Towell and J.W. Shavlik, The extraction of refined rules from\nknowledge-based neural networks, Machine Learning, 13(1) (1993)\n71\u2013101.\n[26] G.G. Towell and J.W. Shavlik. Knowledge-based artificial neural\nnetworks. Artificial Intelligence, 70(1-2):119-165, 1994.\n[27] P.J. Werbos, Backpropagation through time; what it does and how to do\nit, Proceedings of the IEEE, 78:1550-1560, 1990.\n[28] R.J. Williams and D. Zipser, A learning algorithm for continually\nrunning fully recurrent neural networks, Neural Computation,\n1(2):270-280, 1989.\n[29] R.J. Williams and D. Zipser, Gradient-based learning algorithms\nfor recurrent networks and their computational complexity, In\nBack-propagation: Theory, Architectures and Applications, 433-486.\n1995."]}

In this paper, we present an optimization technique or a learning algorithm using the hybrid architecture by combining the most popular sequence recognition models such as Recurrent Neural Networks (RNNs) and Hidden Markov models (HMMs). In order to improve the sequence/pattern recognition/classification performance by applying a hybrid/neural symbolic approach, a gradient descent learning algorithm is developed using the Real Time Recurrent Learning of Recurrent Neural Network for processing the knowledge represented in trained Hidden Markov Models. The developed hybrid algorithm is implemented on automata theory as a sample test beds and the performance of the designed algorithm is demonstrated and evaluated on learning the deterministic finite state automata.