It’s stunning how a few of the primary topics in machine studying are nonetheless unknown by researchers and regardless of being basic and customary to make use of, appear to be mysterious. It’s a enjoyable factor about machine studying that we construct issues that work after which determine why they work in any respect!
Right here, I goal to analyze the unknown territory in some machine studying ideas so as to present whereas these concepts can appear primary, in actuality, they’re constructed by layers upon layers of abstraction. This helps us to follow questioning the depth of our information.
In this text, we discover a number of key phenomena in deep studying that problem our conventional understanding of neural networks.
- We begin with Batch Normalization and its underlying mechanisms that stay not absolutely understood.
- We study the counterintuitive commentary that overparameterized fashions typically generalize higher, contradicting the classical machine studying theories.
- We discover the implicit regularization results of gradient descent, which appear to naturally bias neural networks in the direction of less complicated, extra…