Downloads provided by UsageCountsPlease grant OpenAIRE to access and update your ORCID works.
This Research product is the result of merged Research products in OpenAIRE.
You have already added 0 works in your ORCID record related to the merged Research product.
You have already added 0 works in your ORCID record related to the merged Research product.
This Research product is the result of merged Research products in OpenAIRE.
You have already added 0 works in your ORCID record related to the merged Research product.
You have already added 0 works in your ORCID record related to the merged Research product.
Grounding Representation Similarity with Statistical Testing
Ding, Frances; Jean-Stanislas Denain; Steinhardt, Jacob;
Ding, Frances; Jean-Stanislas Denain; Steinhardt, Jacob;
Grounding Representation Similarity with Statistical Testing
To understand neural network behavior, recent works quantitatively compare different networks' learned representations using canonical correlation analysis (CCA), centered kernel alignment (CKA), and other dissimilarity measures. Unfortunately, these widely used measures often disagree on fundamental observations, such as whether deep networks differing only in random initialization learn similar representations. These disagreements raise the question: which, if any, of these dissimilarity measures should we believe? We provide a framework to ground this question through a concrete test: measures should have sensitivity to changes that affect functional behavior, and specificity against changes that do not. We quantify this through a variety of functional behaviors including probing accuracy and robustness to distribution shift, and examine changes such as varying random initialization and deleting principal components. We find that current metrics exhibit different weaknesses, note that a classical baseline performs surprisingly well, and highlight settings where all metrics appear to fail, thus providing a challenge set for further improvement.
Comment: Accepted at NeurIPS 2021. 10 pages, 3 figures
- Johns Hopkins University United States
- University of California, Berkeley United States
arXiv: Computer Science::Digital Libraries
Machine Learning (cs.LG), Machine Learning (stat.ML), FOS: Computer and information sciences, Computer Science - Machine Learning, Statistics - Machine Learning
Machine Learning (cs.LG), Machine Learning (stat.ML), FOS: Computer and information sciences, Computer Science - Machine Learning, Statistics - Machine Learning
arXiv: Computer Science::Digital Libraries
4115
[1] J. Adebayo, J. Gilmer, M. Muelly, I. Goodfellow, M. Hardt, and B. Kim. Sanity checks for saliency maps. Advances in Neural Information Processing Systems, 31:9505-9515, 2018.
[2] G. Alain and Y. Bengio. Understanding intermediate layers using linear classifier probes, 2018.
[3] S. Arora, Y. Liang, and T. Ma. A simple but tough-to-beat baseline for sentence embeddings. 2017.
[4] Y. Belinkov, N. Durrani, F. Dalvi, H. Sajjad, and J. Glass. What do neural machine translation models learn about morphology? In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 861-872, 2017. [OpenAIRE]
[5] X. Chen, C. Liu, B. Li, K. Lu, and D. Song. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526, 2017.
[6] A. Conneau, G. Lample, M. Ranzato, L. Denoyer, and H. Jégou. Word translation without parallel data, 2018.
[7] A. D'Amour, K. Heller, D. Moldovan, B. Adlam, B. Alipanahi, A. Beutel, C. Chen, J. Deaton, J. Eisenstein, M. D. Hoffman, et al. Underspecification presents challenges for credibility in modern machine learning. arXiv preprint arXiv:2011.03395, 2020.
[8] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
[9] Y. Feng, R. Zhai, D. He, L. Wang, and B. Dong. Transferred discrepancy: Quantifying the difference between representations. arXiv preprint arXiv:2007.12446, 2020.
[10] J. Frankle and M. Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. In International Conference on Learning Representations, 2018. [OpenAIRE]
citations This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).0 popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.Average influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).Average impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.Average views 117 downloads 40 citations This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).0 popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.Average influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).Average impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.Average - 117views40downloads
popularity
Popularity provided by BIP!
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
Popularity provided by BIP!influenceInfluence provided by BIP!
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
Influence provided by BIP!impulseImpulse provided by BIP!
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
Impulse provided by BIP!viewsViews provided by UsageCounts
Views provided by UsageCountsdownloadsDownloads provided by UsageCounts
Downloads provided by UsageCounts Average
Average
Average
117
40
Funded by
NSF| Collaborative Research: Transferable, Hierarchical, Expressive, Optimal, Robust, Interpretable Networks, NSF| Graduate Research Fellowship Program (GRFP)
Project
- National Science Foundation (NSF)
- 2031985
- Directorate for Mathematical & Physical Sciences | Division of Mathematical Sciences
Project
- National Science Foundation (NSF)
- 1752814
- Directorate for Education & Human Resources | Division of Graduate Education
Do the share buttons not appear? Please make sure, any blocking addon is disabled, and then reload the page.
- Johns Hopkins University United States
- University of California, Berkeley United States
To understand neural network behavior, recent works quantitatively compare different networks' learned representations using canonical correlation analysis (CCA), centered kernel alignment (CKA), and other dissimilarity measures. Unfortunately, these widely used measures often disagree on fundamental observations, such as whether deep networks differing only in random initialization learn similar representations. These disagreements raise the question: which, if any, of these dissimilarity measures should we believe? We provide a framework to ground this question through a concrete test: measures should have sensitivity to changes that affect functional behavior, and specificity against changes that do not. We quantify this through a variety of functional behaviors including probing accuracy and robustness to distribution shift, and examine changes such as varying random initialization and deleting principal components. We find that current metrics exhibit different weaknesses, note that a classical baseline performs surprisingly well, and highlight settings where all metrics appear to fail, thus providing a challenge set for further improvement.
Comment: Accepted at NeurIPS 2021. 10 pages, 3 figures
Do the share buttons not appear? Please make sure, any blocking addon is disabled, and then reload the page.