flâneur

[2002.05202] GLU Variants Improve Transformer

arxiv.org · 552 words · saved by 1 readers

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs. arXiv Operational Status Get status notifications via email or slack

[2002.05202] GLU Variants Improve Transformer Skip to main content arXiv is now an independent nonprofit! Learn more × Search arXiv Press Enter to search · Advanced search --> Computer Science > Machine Learning arXiv:2002.05202 (cs) [Submitted on 12 Feb 2020] Title: GLU Variants Improve Transformer Authors: Noam Shazeer View a PDF of the paper titled GLU Variants Improve Transformer, by Noam Shazeer View PDF Abstract: Gated Linear Units ( arXiv:1612.08083 ) consist of the component-wise product of two linear projections, one of which is first passed through a sigmoid function. Va

related reading