Google Research tests a 'sleep' phase so LLMs keep learning after deployment

Google Research described a framework where a model alternates between normal use and an offline 'sleep' phase. During sleep it adds new low-rank expert blocks, distils what it held in short-term context into them, and trains on self-generated practice examples, aiming to learn new facts without forgetting old ones. The authors report near-zero forgetting when learning unseen languages such as Manchu and Kalamang in stages, near-perfect results on the BABILong long-context test up to 10M tokens, and 79.2% and 69.0% on AIME 24 and 25. This is a research result reported by the authors, not a product. On Hacker News (15 top-level comments when we read it) there is no consensus: some praised the clear write-up, while sceptics asked whether it holds for much larger models and objected to the neuroscience framing. Sorted take: interesting direction, nothing to change in your setup yet.