flâneur

May 2020 News · Gwern.net

gwern.net · 2,457 words · saved by 1 readers

May 2020 Gwern.net newsletter: GPT-3 scaling, implications, deep theory; anime GAN updates, and 1 book review. M ay 2020’s Gwern.net⁠ newsletter⁠ is now out; previous, April 2020⁠ (archives⁠). This is a collation of links and summary of major changes, overlapping with my Changelog⁠; brought to you by my donors on Patreon⁠. Ganbooru prototype: released: ⁠“Making Anime With BigGAN § Danbooru2019+e621 256px BigGAN ”⁠, ⁠Gwern⁠2019 (Danbooru AI, BigGAN, Python, tutorial; ⁠backlinks⁠, ⁠bibliography⁠): Experiments in using BigGAN to generate anime faces and whole anime images; semi-successful. Release of a 256px ⁠BigGAN⁠ model trained on Danbooru2019 & e621. This is a prototype model testing our ability to train a BigGAN stably for hundreds of thousands of iterations on a TPU-256⁠ pod on 3 million+ anime/illustration images. While the generated samples are far from ‘photorealistic’, they serve as proof of concept that—unlike our failed StyleGAN2⁠ scaling experiments—BigGAN can successfully mo

--- title: "May 2020 News" description: "May 2020 Gwern.net newsletter: GPT-3 scaling, implications, deep theory; anime GAN updates, and 1 book review." thumbnail: /doc/ai/nn/transformer/gpt/2020-brown-gpt3-figure13-meanperformancescalingcurve.png thumbnail-text: "Figure 1.3 from Brown et al 2020 (OpenAI, GPT-3), showing roughly log-scaling of GPT-3 parameter/compute size vs benchmark performance on all text/natural language benchmarks test." created: 2019-12-26 status: finished confidence: log css-extension: dropcaps-de-zs backlink: False ... May 2020's [Gwern.net](/newsletter/2020/05 "'May 2

related reading