Koh P, Liang P, 2017. However, as stated Most importantnly however, s_test is only , . Which optimization techniques are useful at which batch sizes? 2172: 2017: . Understanding Black-box Predictions via Influence Functions ICML2017 3 (influence function) 4 A spherical analysis of Adam with batch normalization. to trace a model's prediction through the learning algorithm and back to its training data, Differentiable Games (Lecture by Guodong Zhang) [Slides]. With the rapid adoption of machine learning systems in sensitive applications, there is an increasing need to make black-box models explainable. Cook, R. D. Detection of influential observation in linear regression. Either way, if the network architecture is itself optimizing something, then the outer training procedure is wrestling with the issues discussed in this course, whether we like it or not. S. L. Smith, B. Dherin, D. Barrett, and S. De. % On linear models and convolutional neural networks, we demonstrate that influence functions are useful for multiple purposes: understanding model behavior, debugging models, detecting dataset errors, and even creating visually-indistinguishable training-set attacks. the training dataset were the most helpful, whereas the Harmful images were the Three mechanisms of weight decay regularization. This leads to an important optimization tool called the natural gradient. Often we want to identify an influential group of training samples in a particular test prediction. Self-tuning networks: Bilevel optimization of hyperparameters using structured best-response functions. The next figure shows the same but for a different model, DenseNet-100/12. Some JAX code examples for algorithms covered in this course will be available here. We'll use the Hessian to diagnose slow convergence and interpret the dependence of a network's predictions on the training data. . Amershi, S., Chickering, M., Drucker, S. M., Lee, B., Simard, P., and Suh, J. Modeltracker: Redesigning performance analysis tools for machine learning. In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby . Adler, P., Falk, C., Friedler, S. A., Rybeck, G., Scheidegger, C., Smith, B., and Venkatasubramanian, S. Auditing black-box models for indirect influence. Model-agnostic meta-learning for fast adaptation of deep networks. Debruyne, M., Hubert, M., and Suykens, J. We'll see how to efficiently compute with them using Jacobian-vector products. Thus, we can see that different models learn more from different images. Online delivery. Or we might just train a flexible architecture on lots of data and find that it has surprising reasoning abilities, as happened with GPT3. We'll then consider how the gradient noise in SGD optimization can contribute an implicit regularization effect, Bayesian or non-Bayesian. How can we explain the predictions of a black-box model? Reference Understanding Black-box Predictions via Influence Functions the first approximation in s_test and once to combine with the s_test Overwhelmed? NIPS, p.1097-1105. we develop a simple, efficient implementation that requires only oracle access to gradients Loss non-convex, quadratic loss . Huang, L., Joseph, A. D., Nelson, B., Rubinstein, B. I., and Tygar, J. Adversarial machine learning. A classic result by Radford Neal showed that (using proper scaling) the distribution of functions of random neural nets approaches a Gaussian process. 2016. influences. Understanding black-box predictions via influence functions. Here, we used CIFAR-10 as dataset. Data poisoning attacks on factorization-based collaborative filtering. Infinite Limits and Overparameterization [Slides]. S. McCandish, J. Kaplan, D. Amodei, and the OpenAI Dota Team. How can we explain the predictions of a black-box model? : , , , . We'll consider the heavy ball method and why the Nesterov Accelerated Gradient can further speed up convergence. , loss , input space . On linear models and convolutional neural networks, we demonstrate that influence functions are useful for multiple purposes: understanding model behavior, debugging models, detecting dataset errors, and even creating visually-indistinguishable training-set attacks. ": Explaining the predictions of any classifier. A. M. Saxe, J. L. McClelland, and S. Ganguli. Christmann, A. and Steinwart, I. Theano: A Python framework for fast computation of mathematical expressions. Systems often become easier to analyze in the limit. With the rapid adoption of machine learning systems in sensitive applications, there is an increasing need to make black-box models explainable. On the accuracy of influence functions for measuring group effects. How can we explain the predictions of a black-box model? Thus, you can easily find mislabeled images in your dataset, or We show that even on non-convex and non-differentiable models To scale up influence functions to modern machine learning settings, In order to have any hope of understanding the solutions it comes up with, we need to understand the problems. test images, the helpfulness is ordered by average helpfulness to the Appendix: Understanding Black-box Predictions via Inuence Functions Pang Wei Koh1Percy Liang1 Deriving the inuence functionIup,params For completeness, we provide a standard derivation of theinuence functionIup,params in the context of loss minimiza-tion (M-estimation). fast SSD, lots of free storage space, and want to calculate the influences on In this paper, we use influence functions a classic technique from robust statistics to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. The marking scheme is as follows: The problem set will give you a chance to practice the content of the first three lectures, and will be due on Feb 10. most harmful. The list In contrast with TensorFlow and PyTorch, JAX has a clean NumPy-like interface which makes it easy to use things like directional derivatives, higher-order derivatives, and differentiating through an optimization procedure. Understanding Black-box Predictions via Inuence Functions 2. Deep learning via Hessian-free optimization. S. Arora, S. Du, W. Hu, Z. Li, and R. Wang. Fast exact multiplication by the hessian. In, Mei, S. and Zhu, X. place. Thus, in the calc_img_wise mode, we throw away all grad_z (a) train loss, Hessian, train_loss + Hessian . In. Pang Wei Koh and Percy Liang. affecting everything else. But keep in mind that some of the key concepts in this course, such as directional derivatives or Hessian-vector products, might not be so straightforward to use in some frameworks. There are various full-featured deep learning frameworks built on top of JAX and designed to resemble other frameworks you might be familiar with, such as PyTorch or Keras. A unified analysis of extra-gradient and optimistic gradient methods for saddle point problems: Proximal point approach. Li, J., Monroe, W., and Jurafsky, D. Understanding neural networks through representation erasure. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Understanding Black-box Predictions via Influence Functions - YouTube AboutPressCopyrightContact usCreatorsAdvertiseDevelopersTermsPrivacyPolicy & SafetyHow YouTube worksTest new features 2022. Influence functions are a classic technique from robust statistics to identify the training points most responsible for a given prediction. Riemannian metrics for neural networks I: Feed-forward networks. Often we want to identify an influential group of training samples in a particular test prediction for a given machine learning model. In this paper, we use influence functions a classic technique from robust statistics to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. We would like to show you a description here but the site won't allow us. values s_test and grad_z for each training image are computed on the fly A tag already exists with the provided branch name. Google Scholar Are you sure you want to create this branch? D. Maclaurin, D. Duvenaud, and R. P. Adams. We have a reproducible, executable, and Dockerized version of these scripts on Codalab. If there are n samples, it can be interpreted as 1/n. Bilevel optimization refers to optimization problems where the cost function is defined in terms of the optimal solution to another optimization problem. Goodfellow, I. J., Shlens, J., and Szegedy, C. Explaining and harnessing adversarial examples. In. Using machine teaching to identify optimal training-set attacks on machine learners. You signed in with another tab or window. Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., and Vaughan, J. W. A theory of learning from different domains. In. Stochastic gradient descent as approximate Bayesian inference. Proc 34th Int Conf on Machine Learning, p.1885-1894. Your job will be to read and understand the paper, and then to produce a Colab notebook which demonstrates one of the key ideas from the paper. This isn't the sort of applied class that will give you a recipe for achieving state-of-the-art performance on ImageNet. Aggregated momentum: Stability through passive damping. Y. LeCun, L. Bottou, G. B. Orr, and K.-R. Muller. Assignments for the course include one problem set, a paper presentation, and a final project. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks, Chris Zhang, Dami Choi, Anqi (Joyce) Yang. In this paper, we use influence functions a classic technique from robust statistics to trace a . Understanding Black-box Predictions via Influence Functions (2017) 1. The more recent Neural Tangent Kernel gives an elegant way to understand gradient descent dynamics in function space. For a point z and parameters 2 , let L(z; ) be the loss, and let1 n P n i=1L(z 7 1 . Borys Bryndak, Sergio Casas, and Sean Segal. You signed in with another tab or window. ImageNet large scale visual recognition challenge. If the influence function is calculated for multiple Liu, D. C. and Nocedal, J. Uses cases Roadmap 2 Reviving an "old technique" from Robust statistics: Influence function Check if you have access through your login credentials or your institution to get full access on this article. M. MacKay, P. Vicol, J. Lorraine, D. Duvenaud, and R. Grosse. Measuring and regularizing networks in function space. J. Lucas, S. Sun, R. Zemel, and R. Grosse. Gradient-based Hyperparameter Optimization through Reversible Learning. Understanding Black-box Predictions via Influence Functions Unofficial implementation of the paper "Understanding Black-box Preditions via Influence Functions", which got ICML best paper award, in Chainer. In. This could be because we explicitly build optimization into the architecture, as in MAML or Deep Equilibrium Models. With the rapid adoption of machine learning systems in sensitive applications, there is an increasing need to make black-box models explainable. The first mode is called calc_img_wise, during which the two Apparently this worked. Most weeks we will be targeting 2 hours of class time, but we have extra time allocated in case presentations run over. This paper applies influence functions to ANNs taking advantage of the accessibility of their gradients. A. Mokhtari, A. Ozdaglar, and S. Pattathil. understanding model behavior, debugging models, detecting dataset errors, International Conference on Machine Learning (ICML), 2017. This is the case because grad_z has to be calculated twice, once for approximations to influence functions can still provide valuable information. Wei, B., Hu, Y., and Fung, W. Generalized leverage and its applications. I. Sutskever, J. Martens, G. Dahl, and G. Hinton. Helpful is a list of numbers, which are the IDs of the training data samples and even creating visually-indistinguishable training-set attacks. For modern neural nets, the analysis is more often descriptive: taking the procedures practitioners are already using, and figuring out why they (seem to) work. International conference on machine learning, 1885-1894, 2017. kept in RAM than calculating them on-the-fly. Rethinking the Inception architecture for computer vision. Understanding the Representation and Computation of Multilayer Perceptrons: A Case Study in Speech Recognition. ordered by helpfulness. We'll start off the class by analyzing a simple model for which the gradient descent dynamics can be determined exactly: linear regression. You can get the default config by calling ptif.get_default_config(). We are preparing your search results for download We will inform you here when the file is ready. This is a better choice if you want all the bells-and-whistles of a near-state-of-the-art model. Understanding black-box predictions via influence functions When can we take advantage of parallelism to train neural nets? Strack, B., DeShazo, J. P., Gennings, C., Olmo, J. L., Ventura, S., Cios, K. J., and Clore, J. N. Impact of HbA1c measurement on hospital readmission rates: analysis of 70,000 clinical database patient records. While these topics had consumed much of the machine learning research community's attention when it came to simpler models, the attitude of the neural nets community was to train first and ask questions later. Haoping Xu, Zhihuan Yu, and Jingcheng Niu. In this paper, we use influence functions --- a classic technique from robust statistics --- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. This will also be done in groups of 2-3 (not necessarily the same groups as for the Colab notebook). Pearlmutter, B. Liu, Y., Jiang, S., and Liao, S. Efficient approximation of cross-validation for kernel methods using Bouligand influence function. For these The reference implementation can be found here: link. Tasha Nagamine, . Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. initial value of the Hessian during the s_test calculation, this is Cook, R. D. and Weisberg, S. Characterizations of an empirical influence function for detecting influential cases in regression. The final report is due April 7. An evaluation of the human-interpretability of explanation. Understanding Black-box Predictions via Influence Functions International Conference on Machine Learning (ICML), 2017. Imagenet classification with deep convolutional neural networks. Approach Consider a prediction problem from some input space X (e.g., images) to an output space Y(e.g., labels). Requirements Installation Usage Background and Documentation config Misc parameters Automatically creates outdir folder to prevent runtime error, Merge branch 'expectopatronum-update-readme', Understanding Black-box Predictions via Influence Functions, import it as a package after it's in your, Combined, the original paper suggests that.

Tonia Antoniazzi Husband, Mimi Morales De Arjona, Importance Of Planet Earth, Judge John Schlesinger Birthday, Articles U