implicit bias
Latest
The Neural Race Reduction: Dynamics of nonlinear representation learning in deep architectures
What is the relationship between task, network architecture, and population activity in nonlinear deep networks? I will describe the Gated Deep Linear Network framework, which schematizes how pathways of information flow impact learning dynamics within an architecture. Because of the gating, these networks can compute nonlinear functions of their input. We derive an exact reduction and, for certain cases, exact solutions to the dynamics of learning. The reduction takes the form of a neural race with an implicit bias towards shared representations, which then govern the model’s ability to systematically generalize, multi-task, and transfer. We show how appropriate network architectures can help factorize and abstract knowledge. Together, these results begin to shed light on the links between architecture, learning dynamics and network performance.
On the implicit bias of SGD in deep learning
Tali's work emphasized the tradeoff between compression and information preservation. In this talk I will explore this theme in the context of deep learning. Artificial neural networks have recently revolutionized the field of machine learning. However, we still do not have sufficient theoretical understanding of how such models can be successfully learned. Two specific questions in this context are: how can neural nets be learned despite the non-convexity of the learning problem, and how can they generalize well despite often having more parameters than training data. I will describe our recent work showing that gradient-descent optimization indeed leads to 'simpler' models, where simplicity is captured by lower weight norm and in some cases clustering of weight vectors. We demonstrate this for several teacher and student architectures, including learning linear teachers with ReLU networks, learning boolean functions and learning convolutional pattern detection architectures.
Blindspot: Hidden Biases of Good People
Mahzarin Banaji and her colleague coined the term “implicit bias” in the mid-1990s to refer to behavior that occurs without conscious awareness. Today, Professor Banaji is Cabot Professor of Social Ethics in the Department of Psychology at Harvard University, a member of the American Academy of Arts and Sciences, the National Academy of Sciences and has received numerous awards for her scientific contributions. The purpose of the seminar, Blindspot: Hidden Biases of Good People, is to reveal the surprising and even perplexing ways in which we make errors in assessing and evaluating others when we recruit and hire, onboard and promote, lead teams, undertake succession planning, and work on behalf of our clients or the public we serve. It is Professor Banaji’s belief that people intend well and that the inconsistency we see, between values and behavior, comes from a lack of awareness. But because implicit bias is pervasive, we must rely on scientific evidence to “outsmart” our minds. If we do so, we will be more likely to reach the life goals we have chosen for ourselves and to serve better the organizations for which we work.
implicit bias coverage
3 items
Share your knowledge
Know something about implicit bias? Help the community by contributing seminars, talks, or research.
Contribute content