Paper shows a model can hide an encrypted skill that tests cannot find
Four researchers built a backdoor that hides an encrypted neural network inside a model's weights. It only decrypts and runs when the model gets a secret trigger. Using a cryptographic method called digital lockers, they argue the hidden skill is provably hard to find or understand, even with full access to the weights. In a small proof of concept on chemical reaction prediction, used as a stand-in for dangerous capabilities, fine-tuning, adding noise to the weights, searching for the trigger and adversarial training all failed to bring it out. The paper is accepted at NeurIPS 2026 and the models are released to help build defences. This is a lab demonstration, not an attack seen in the wild.
On Hacker News (about 140 comments), there is no consensus. One camp worries about open-weight models from untrusted sources carrying sleeper triggers, and some hope it boosts demand for fully open models with inspectable training data. Another camp says the headline misleads: this is about a builder deliberately hiding something with encryption, not models choosing to hide skills. Skeptics called it a neat demo with little to act on, and one said such a model would be easy to spot.
Sorted take: this mostly matters if you run open-weight models yourself. Download from the original publisher, check file hashes, and give a local model no more tool access than you would give an unknown app.
Why it matters: Capability tests and red-team reports cannot prove a model has nothing hidden. If you pick open-weight models for work, where they came from matters as much as their benchmark scores.
arXiv ↗Discussion on Hacker News ↗AI-assisted, reviewed by Sorted AI
▲ 0
0 comments