Inferock-bench is a local proxy that sits between your app and OpenAI, Anthropic, Gemini, or OpenRouter shaped calls. It captures per-call token usage, failures, and retries, then generates an independent receipt showing what you were billed and how much you're actually overpaying for.
This is the 2nd launch from Inferock. View more
Inferock
Launching today
Get free signup credits to use all available models on Inferock.
Bring your own keys: run your OpenAI, Claude or Gemini accounts, see where calls fail and what they cost, and get credits back on eligible failed calls.
Serverless: pay-per-token LLM inference. Get signup credits to start using.
Free Options
Launch Team / Built With
Tines The single, secure environment for agents, apps, and automations.
Promoted
Hi Product Hunt community, I’m Bharath, founder of Inferock.
Every team running LLMs in production pays for calls that came back empty, truncated, timed out, or returned broken JSON. You usually find out from a user, and you can’t prove it to anyone.
Inferock sits between your app and the provider and measures every call against a published baseline for quality, speed and cost. Each call gets a receipt that Inferock produces itself, so it doesn’t depend on the provider’s word.
Two ways to use it:
- Inferock Watch: bring your own OpenAI, Anthropic or Gemini keys. Free. You get the receipts and a monthly loss report. Enroll in the credit program and objectively broken calls accrue credits you can spend on Serve.
- Inferock Serve: serverless, pay-per-token inference through us. This is where your credits get spent.
How credits work: credits are non-cash, expire after 12 months, and are capped at $1,000 in the wallet and $1,000 applied per month. They cover verified, billed, unrecovered objective failures: downtime and timeouts, empty or truncated output, invalid structured output, billing and cache anomalies.
Tell us what the receipt should show that it doesn’t yet.
One thing we’ve learned while building this is that a failed LLM call is not always obvious.
Sometimes the request technically succeeds, but the output is incomplete, too slow, or unusable for the app.
That’s why we’re trying to make the receipt useful beyond just showing pass or fail. Curious what details you’d want to see on it when something goes wrong.
Thanks for hunting @fmerian!
@bharath_koneti This is super cool! Solving a real problem with cost and quality visibility and control, with broad provider support.
@justinmahood Thank you, really appreciate it! Cost and quality visibility you can act on is exactly the goal.
This is pretty neat. Can you expand on the Cloud RUn architecture? I see that you host Inferlock.ai inference router but are you using services, jobs, some mix?
@cloudalchemist We use Cloud Run Services for the live router, not Jobs. The router is stateless and I/O-heavy: it validates the request, applies tenant routing and spending policies, proxies to the selected model provider, and streams the result back to the client. We keep one production instance warm and scale out under load.
@rukhsar_amjad Thank you! Invalid structured output is one of the failure types Inferock tracks on each call. The call record doesn't show the exact point. We didn't want to hold any PII's in the failure set!
@nitish_mishra_vaani Thank you. We are quantifying failures across https://inferock.ai/baseline/ in your BYOK keys. We are showing that you shouldn't be receiving those failures and supporting with credits which you can use towards our platform today!.
We are not showing the exact failure points today. we'll take this as a feedback and include it in the coming releases!
It's not clear what should happen after that, should we contact Anthropic or openai for a refund for this calls?
Great question @bengeekly For Inferock Watch, we issue non-cash, expiring service credits for verified, billed, unrecovered failures.
Refusals, unbilled calls, and fully recovered costs earn no credit.
The wallet cap is $1,000 and the applied cap is $1,000 per calendar month; the service charge stays cash.
I love that it sits as a local proxy rather than requiring us to pipe sensitive request data tthrough a third party cloud dashboard.
@sandy_leventh Thank you, keeping sensitive request data under your control matters a lot to us too. Our measurement tool runs as a local proxy and is source available (github.com/inferock/inferock-bench).