Why one developer routes all his AI calls through his own proxy

In a short blog post, developer Jared Siegel argues that the usage charts in AI chat apps are not enough to make decisions with, and that anyone using AI seriously should run their own LLM proxy: a small server that sits between your tools and the AI providers. He lists what it gives you: one endpoint for many providers and models, a log of which key called which model from which tool, when and with what result, separate virtual keys so your main API key is not pasted into every agent and script, and the ability to switch provider without setting everything up again. He says it lets him find the cause of a cache miss in about 20 seconds. He deliberately does not name the proxy he uses. On Hacker News (292 points, 208 comments), 6 of the 41 top-level commenters run or recommend a proxy or gateway, naming LiteLLM, OpenRouter, Tailscale's Aperture, Bifrost and two they built themselves, and one more agrees but calls LiteLLM and similar tools buggy. Twelve describe getting the same visibility without a proxy, mostly through OpenTelemetry traces, session log files, their own agent harness or a dedicated app, and one warns that a proxy adds a single point of failure. Four debate whether pay-per-use API pricing is worth it over a flat subscription, seven wave the idea off as overkill or reject AI tools altogether, six ask questions, including which proxy the author uses, and five make side points. No consensus. Sorted take: If more than one person or agent in your team calls AI APIs, you need some record of who spent what and why; a proxy is one way to get it, OpenTelemetry tracing is another, and either beats guessing from the monthly bill.