Model-Agnostic Interpretability of Machine Learning

Preprint English OPEN
Ribeiro, Marco Tulio ; Singh, Sameer ; Guestrin, Carlos (2016)
  • Subject: Statistics - Machine Learning | Computer Science - Learning

Understanding why machine learning models behave the way they do empowers both system designers and end-users in many ways: in model selection, feature engineering, in order to trust and act upon the predictions, and in more intuitive user interfaces. Thus, interpretability has become a vital concern in machine learning, and work in the area of interpretable models has found renewed interest. In some applications, such models are as accurate as non-interpretable ones, and thus are preferred for their transparency. Even when they are not accurate, they may still be preferred when interpretability is of paramount importance. However, restricting machine learning to interpretable models is often a severe limitation. In this paper we argue for explaining machine learning predictions using model-agnostic approaches. By treating the machine learning models as black-box functions, these approaches provide crucial flexibility in the choice of models, explanations, and representations, improving debugging, comparison, and interfaces for a variety of users and models. We also outline the main challenges for such methods, and review a recently-introduced model-agnostic explanation approach (LIME) that addresses these challenges.
  • References (21)
    21 references, page 1 of 3

    Baehrens, David, Schroeter, Timon, Harmeling, Stefan, Kawanabe, Motoaki, Hansen, Katja, and Mu¨ller, KlausRobert. How to explain individual classification decisions. Journal of Machine Learning Research, 11, 2010.

    Caruana, Rich, Lou, Yin, Gehrke, Johannes, Koch, Paul, Sturm, Marc, and Elhadad, Noemie. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Knowledge Discovery and Data Mining (KDD), 2015.

    Craven, Mark W and Shavlik, Jude W. Extracting treestructured representations of trained networks. Advances in neural information processing systems, pp. 24-30, 1996.

    Druck, Gregory, Mann, Gideon, and McCallum, Andrew. Learning from labeled features using generalized expectation criteria. In ACM SIGIR conference on Research and development in information retrieval, pp. 595-602. ACM, 2008.

    Freitas, Alex A. Comprehensible classification models: A position paper. SIGKDD Explor. Newsl., 15(1):1-10, March 2014. ISSN 1931-0145.

    Kim, Been, Rudin, Cynthia, and Shah, Julie A. The bayesian case model: A generative approach for case-based reasoning and prototype classification. In Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., and Weinberger, K.Q. (eds.), Advances in Neural Information Processing Systems 27, pp. 1952-1960. Curran Associates, Inc., 2014.

    Kim, Been, Glassman, Elena, Johnson, Brittney, and Shah, Julie. ibcm: Interactive bayesian case model empowering humans via intuitive interaction. 2015.

    Krause, Josua, Perer, Adam, and Ng, Kenney. Interacting with predictions: Visual inspection of black-box machine learning models. 2016.

    Letham, Benjamin, Rudin, Cynthia, McCormick, Tyler H., and Madigan, David. Interpretable classifiers using rules and bayesian analysis: Building a better stroke prediction model. Annals of Applied Statistics, 2015.

    Mikolov, Tomas, Sutskever, Ilya, Chen, Kai, Corrado, Greg S, and Dean, Jeff. Distributed representations of words and phrases and their compositionality. In Neural Information Processing Systems (NIPS). 2013.

  • Similar Research Results (3)
  • Metrics
    No metrics available
Share - Bookmark