Fast relational learning using bottom clause propositionalization with artificial neural networks

Article English OPEN
França, M. V. M. ; Zaverucha, G. ; Garcez, A. (2014)

Relational learning can be described as the task of learning first-order logic rules from examples. It has enabled a number of new machine learning applications, e.g. graph mining and link analysis. Inductive Logic Programming (ILP) performs relational learning either directly by manipulating first-order rules or through propositionalization, which translates the relational task into an attribute-value learning task by representing subsets of relations as features. In this paper, we introduce a fast method and system for relational learning based on a novel propositionalization called Bottom Clause Propositionalization (BCP). Bottom clauses are boundaries in the hypothesis search space used by ILP systems Progol and Aleph. Bottom clauses carry semantic meaning and can be mapped directly onto numerical vectors, simplifying the feature extraction process. We have integrated BCP with a well-known neural-symbolic system, C-IL2P, to perform learning from numerical vectors. C-IL2P uses background knowledge in the form of propositional logic programs to build a neural network. The integrated system, which we call CILP++, handles first-order logic knowledge and is available for download from Sourceforge. We have evaluated CILP++ on seven ILP datasets, comparing results with Aleph and a well-known propositionalization method, RSD. The results show that CILP++ can achieve accuracy comparable to Aleph, while being generally faster, BCP achieved statistically significant improvement in accuracy in comparison with RSD when running with a neural network, but BCP and RSD perform similarly when running with C4.5. We have also extended CILP++ to include a statistical feature selection method, mRMR, with preliminary results indicating that a reduction of more than 90 % of features can be achieved with a small loss of accuracy.
  • References (55)
    55 references, page 1 of 6

    Bain, M., & Muggleton, S. (1994). Learning optimal chess strategies. Machine Intelligence, 13, 291-309.

    Basilio, R., Zaverucha, G., and Barbosa, V. (2001). Learning logic programs with neural networks. In Proc. ILP, LNAI 2157: 402-408. Springer.

    Caruana, R., Lawrence, S., & Giles, C. L. (2000). Overfitting in neural nets: backpropagation, conjugate gradient, and early stopping. In Proc. NIPS, 13: 402-408. MIT Press.

    Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16 (1): 321- 357.

    Clark, P., & Niblett, T. 1989. The CN2 induction algorithm. Machine Learning, 3: 261-283.

    Copelli, M., Eichhorn, R., Kinouchi, O., Biehl, M., Simonetti, R., Riegler, P., & Caticha, N. (1997). Noise robustness in multilayer neural networks. EPL (Europhysics Letters), 37 (6): 427-432.

    Craven, M., & Shavlik, J. W. (1995). Extracting tree-structured representations of trained networks. In Proc. NIPS, 9: 24-30. Cambridge, MA, USA: The MIT Press.

    Davis, J., Burnside, E. S., Dutra, I. C., Page, D., & Costa, V. S. (2005). An integrated approach to learning Bayesian networks of rules. In Proc. ECML, LNAI 3720: 84-95. BerlinHeidelberg, Germany: Springer.

    De Raedt, L. (2008). Logical and relational learning. Berlin-Heidelberg, Germany: Springer.

    De Raedt, L., Frasconi, P., Kersting, K., & Muggleton, S. (2008). Probabilistic inductive logic programming. LNAI 4911. Berlin-Heidelberg, Germany: Springer.

  • Software (1)
  • Metrics
    views in OpenAIRE
    views in local repository
    downloads in local repository

    The information is available from the following content providers:

    From Number Of Views Number Of Downloads
    City Research Online - IRUS-UK 0 296
Share - Bookmark