✳flâneur — a map of the web's best reading

LLM Daydreaming · Gwern.net

gwern.net · 2,285 words · saved by 12 readers

Proposal & discussion of how default mode networks for LLMs are an example of missing capabilities for search and novelty in contemporary AI systems. De­spite im­pres­sive ca­pa­bil­i­ties, large lan­guage mod­els have yet to pro­duce a gen­uine break­through. The puz­zle is why. A rea­son may be that they lack some fun­da­men­tal as­pects of human thought: they are frozen, un­able to learn from ex­pe­ri­ence, and they have no “de­fault mode” for back­ground pro­cess­ing, a source of spon­ta­neous human in­sight. To solve this, I pro­pose a day-dreaming loop (DDL): a back­ground process that con­tin­u­ously sam­ples pairs of con­cepts from mem­ory. A gen­er­a­tor model ex­plores non-obvious links be­tween them, and a critic model fil­ters the re­sults for gen­uinely valu­able ideas. These dis­cov­er­ies are fed back into the sys­tem’s mem­ory, cre­at­ing a com­pound­ing feed­back loop where new ideas them­selves be­come seeds for fu­ture com­bi­na­tions. The cost of this process—a “day

--- title: LLM Daydreaming description: "Proposal & discussion of how default mode networks for LLMs are an example of missing capabilities for search and novelty in contemporary AI systems." created: 2025-07-12 modified: 2025-07-14 status: finished confidence: possible importance: 6 css-extension: dropcaps-kanzlei ... > Despite impressive capabilities, large language models have yet to produce a genuine breakthrough. > The puzzle is why. > > A reason may be that they lack some fundamental aspects of human thought: they are frozen, unable to learn from experience, and they have no "default mod

Explore this link on the map →

saved by

related reading