Teraflop-scale Incremental Machine Learning

Preprint English OPEN
Özkural, Eray (2011)
  • Subject: Computer Science - Artificial Intelligence

We propose a long-term memory design for artificial general intelligence based on Solomonoff's incremental machine learning methods. We use R5RS Scheme and its standard library with a few omissions as the reference machine. We introduce a Levin Search variant based on Stochastic Context Free Grammar together with four synergistic update algorithms that use the same grammar as a guiding probability distribution of programs. The update algorithms include adjusting production probabilities, re-using previous solutions, learning programming idioms and discovery of frequent subprograms. Experiments with two training sequences demonstrate that our approach to incremental learning is effective.
  • References (16)
    16 references, page 1 of 2

    1. Solomonoff, R.J.: A system for incremental learning based on algorithmic probability. In: Proceedings of the Sixth Israeli Conference on Artificial Intelligence, Tel Aviv, Israel (1989) 515-527

    2. Hutter, M.: The fastest and shortest algorithm for all well-defined problems. International Journal of Foundations of Computer Science 13(3) (2002) 431-443

    3. Schmidhuber, J.: Ultimate cognition `a la G¨odel. Cognitive Computation 1(2) (2009) 177-193

    4. Schmidhuber, J.: Optimal ordered problem solver. Machine Learning 54 (2004) 211-256

    5. R. Cilibrasi, P.V.: Clustering by compression. Technical report, CWI (2003)

    6. Solomonoff, R.J.: Progress in incremental machine learning. In: NIPS Workshop on Universal Learning Algorithms and Optimal Search. (2002)

    7. Solomonoff, R.J.: Algorithmic probability: Theory and applications. In Dehmer, M., Emmert-Streib, F., eds.: Information Theory and Statistical Learning, Springer Science+Business Media, N.Y. (2009) 1-23

    8. Schmidhuber, J., Zhao, J., Wiering, M.: Shifting inductive bias with successstory algorithm, adaptive levin search, and incremental self-improvement. Machine Learning 28(1) (1997) 105-130

    9. Richard Kelsey, William Clinger, J.R.: Revised5 report on the algorithmic language scheme. Higher-Order and Symbolic Computation 11(1) (1998) (editors).

    10. Levin, L.A.: Universal sequential search problems. Problems of Information Transmission 9(3) (1973) 265-266

  • Similar Research Results (2)
  • Metrics
    No metrics available
Share - Bookmark