The number that lies

A creative tool generated 100 assets in a month for US$40: US$0.40 per creative, according to the model dashboard. Customers published only 20 of them. The real cost is US$2 per published creative, five times more, and the gap hides in retries, tools and human review that nobody added up.

What Holdy tracks

  • Every task through to a confirmed outcome in your customer's workflow, not until the agent says it's done.
  • What customers accept, edit, publish or discard, and how much rework each output needed.
  • Model, retries, tools and human review added up per useful result, per account.
  • Accounts where the AI stalls before the end, opened as an opportunity with the evidence and a written plan.

After the finding

Every opportunity comes with the plan written: ask for the missing context before running the agent, change a screen's default, nudge accounts that spend without results. The change ships as an in-product hint through the SDK, a message, or a draft pull request on GitHub, always with part of the accounts held out, and Holdy shows whether cost per useful result went down and whether conversion and MRR moved.

Set up in minutes

  • Run npx @holdyapp/install in your repo: it installs the SDK and wires the MCP into Claude Code or Cursor.
  • Already on PostHog or Mixpanel? Connect it and Holdy pulls your history, so day one already has a sample.
  • Your coding agent can read opportunities through the Holdy MCP and implement the plan.

Frequently asked questions

What do I need to instrument?

Model calls and the outcome of each task: generated, accepted, edited, published, discarded. The SDK docs list the events, and npx @holdyapp/install helps you wire them.

Does Holdy read what the AI generates?

No. It reads events and numbers (cost, retries, outcome), never the generated text or image.

How much does it cost?

Holdy charges per active account per month: each of your customers, company or workspace, with at least one event in the last 30 days counts once, no matter how many people it has. Launch is US$99 per month up to 200 active accounts and Growth is US$299 up to 1,000. Seats, events and integrations are not charged. Every plan starts with a 14-day free trial, no card required.

How do I know a change worked?

Each action leaves part of the accounts out, at random. Holdy compares both groups on the behavior the action targets and, when you send plan, MRR and status per account, on who converted and who churned. A money figure only shows up once the difference holds; before that, Holdy says it can't tell yet.

See also