​​What learning algorithm is in-context learning? Investigations with linear models | OpenReview

We prove that the transformers can implement learning algorithms for linear models based e.g gradient descent, then observe they closely match the predictors of known algorithms, transitioning...

saved by

• Nick Jiang