A study of pattern recovery in recurrent correlation associative memories

Article English OPEN
Hancock, E.R. ; Wilson, R.C. (2003)

In this paper, we analyze the recurrent correlation associative memory (RCAM) model of Chiueh and Goodman. This is an associative memory in which stored binary memory patterns are recalled via an iterative update rule. The update of the individual pattern-bits is controlled by an excitation function, which takes as its arguement the inner product between the stored memory patterns and the input patterns. Our contribution is to analyze the dynamics of pattern recall when the input patterns are corrupted by noise of a relatively unrestricted class. We make three contributions. First, we show how to identify the excitation function which maximizes the separation (the Fisher discriminant) between the uncorrupted realization of the noisy input pattern and the remaining patterns residing in the memory. Moreover, we show that the excitation function which gives maximum separation is exponential when the input bit-errors follow a binomial distribution. Our second contribution is to develop an expression for the expectation value of bit-error probability on the input pattern after one iteration. We show how to identify the excitation function which minimizes the bit-error probability. However, there is no closed-form solution and the excitation function must be recovered numerically. The relationship between the excitation functions which result from the two different approaches is examined for a binomial distribution of bit-errors. The final contribution is to develop a semiempirical approach to the modeling of the dynamics of the RCAM. This provides us with a numerical means of predicting the recall error rate of the memory. It also allows us to develop an expression for the storage capacity for a given recall error rate.
  • References (29)
    29 references, page 1 of 3

    [1] K. Steinbuch, “Das lern matrix,” Kybernetik, vol. 1, p. 36, 1961.

    [2] D. J. Willshaw, Nature, vol. 222, p. 960, 1969.

    [3] J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” in Proc. Nat. Academy Science USA, vol. 79, 1982, pp. 2554-2558.

    [4] E. Gardner, “Structure of metastable states in the Hopfield model,” J. Phys. A, vol. 19, no. 16, pp. 1047-1052, 1986.

    [5] T. D. Chiueh and R. M. Goodman, “Recurrent correlation associative memories,” IEEE Trans. Neural Networks, vol. 2, pp. 275-284, Mar. 1991.

    [6] , “VLSI implementation of a high-capacity neural network associative memory,” in Advances in Neural Information Processing Systems 2, D. S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1990, pp. 793-800.

    [7] C. C. Wang and H. S. Don, “An analysis of high-capacity discrete exponential BAM,” IEEE Trans. Neural Networks, vol. 6, pp. 492-496, Mar. 1995.

    [8] T. D. Chiueh and H. K. Tsai, “Multi-valued associative memories based on recurrent networks,” IEEE Trans. Neural Networks, vol. 4, pp. 364-366, Mar. 1993.

    [9] P. Kanerva, Sparse Distributed Memory. Cambridge, MA: MIT Press, 1988.

    [10] D. Milun and D. Sher, “Improving sampled probability distributions for Markov random fields,” Pattern Recognition Lett., vol. 14, pp. 781-788, 1993.

  • Similar Research Results (2)
  • Metrics
    0
    views in OpenAIRE
    0
    views in local repository
    22
    downloads in local repository

    The information is available from the following content providers:

    From Number Of Views Number Of Downloads
    White Rose Research Online - IRUS-UK 0 22
Share - Bookmark