Overparameterized but generalized ~ Neural Network | by Purvanshi Mehta | Towards Data Science

Deep learning has evolved from just being parameterized non linear functions to being used in major computer vision and natural language processing tasks. The piecewise non linear networks are able to form non trivial representations of data. Though, these networks have been highly successful there are many gaps between their understanding of why they perform so well by finding near optimal solutions to problems. These models give 0 training error, therefore highly overfitting the training data but are still able to give good test performance. This benign overfitting appears to contradict accepted statistical wisdom, which insists on a trade-off between the complexity of a model and its fit to the data. Therefore it becomes necessary to examine this interesting phenomenon to fill in the gaps. Suppose our task is to map a set of data points X to their labels Y. We define a function f: X->Y. The underlying assumption here is that the data is i.i.d which means each data point was generate

Deep learning has evolved from just being parameterized non linear functions to being used in major computer vision and natural language processing tasks. The piecewise non linear networks are able to form non trivial representations of data. Though, these networks have been highly successful there are many gaps between their understanding of why they perform so well by finding near optimal solutions to problems. These models give 0 training error, therefore highly overfitting the training data but are still able to give good test performance. This benign overfitting appears to contradict acce