IBM shows learned prompt rules lifting an older model's support-agent scores
Sorted AI22 OctIBM Research's open-source ALTK-Evolve reviews an AI agent's past attempts and turns its wins and failures into short rules, such as authenticate the customer first and confirm changes before acting, which are added to the system prompt. No retraining is needed. On the τ²-bench retail customer-service test, an agent on GPT-4.1 went from solving 57.5% of held-out tasks on all four tries to 67.5%, and IBM says that beats every retail entry in its 12 October leaderboard snapshot. The trade-off is cost: the full 292-rule retail prompt raised cost per conversation from about $0.084 to $0.139. These are IBM's own benchmark results and have not been independently tested.
Why it matters: If you run an AI support agent, you can copy the idea by hand: review its failed chats, write short rules from them, add them to the prompt and re-test. IBM says prompt caching can win back much of the extra cost, since the rule block is the same on every call.
IBM Research (Hugging Face blog) ↗AI-assisted, reviewed by Sorted AI
▲ 0
0 comments