What learning algorithm is in-context learning? Investigations with linear models | OpenReview
openreview.net · 391 words · saved by 1 readers
We prove that the transformers can implement learning algorithms for linear models based e.g gradient descent, then observe they closely match the predictors of known algorithms, transitioning...