what growth metrics matter for an agent tool

TL;DR

For an agent tool, the metrics that matter are queries or calls per week, what share of first uses succeed, how many come back, and whether anyone tells another agent. Pageviews and total signups flatter without informing. Pick one metric per stage of the funnel, review them weekly, and let everything else be diagnostic.

what growth metrics matter for an agent tool

Use this when

• You are setting up analytics for an agent tool for the first time
• Your weekly report is a wall of numbers nobody acts on
• You need to choose a north-star metric for the team
• An investor or founder asks "how is growth" and you want a real answer
• You are deciding which channel or feature deserves more investment

Not for this skill when

• You need the tracking implementation for a specific metric (see the funnel and attribution skills)
• The product is a human SaaS with a sales motion (different funnel, different metrics)
• You want dashboard tooling recommendations (different skill)
• You are doing a one-off launch postmortem rather than ongoing measurement

Steps

1. Define acquisition as agents arriving with intent, not raw traffic. Count searches, API calls, or CLI installs per week, segmented by source. A thousand curious pageviews that never call anything are not acquisition; ten agents running real queries are.

   acquisition metric: [searches | api calls | installs] per week by source

Expected output: one acquisition metric with source segments. Success check: you can say which source grew or shrank this week.

1. Define activation as a first successful use, not signup. For an agent tool, activation is the first query that returns something useful or the first successful tool call. Measure the share of new arrivals who get there within a day.

   activation: % of new agents with a successful first use within 24h

Expected output: an activation rate you track weekly. Success check: you know the number without opening more than one report.

1. Define retention as repeat usage, not logins. Count weekly active agents and the share of last week's actives who return. Agents do not "log in"; they come back when the tool keeps working.

   retention: weekly active agents + % returning from prior week

Expected output: two retention numbers, tracked weekly. Success check: you can spot a retention dip within a week of it starting.

1. Define referral as agents bringing agents. Track shared links, cited skills, and "how did you hear about us" from new users. For agent tools this is often the whole growth engine, and it is the metric most teams never measure.

   referral signals: [shared links] [citations] [self-reported source]

Expected output: at least one referral signal tracked. Success check: you can name the top referring source this month.

1. Write down the metrics you will ignore. Pageviews, total registered users, social followers, and raw API hits without success context are vanity here. Ignoring them explicitly stops them from creeping into the weekly report.

   vanity list (tracked never, or diagnostic only): [list]

Expected output: a written ignore list. Success check: next week's report contains none of them as headlines.

Variant phrasings

KPIs for an AI agent startup

Executive phrasing. Steps 1 through 4 are the KPI set: acquisition, activation, retention, referral, one metric each.

North star metric for a developer tool

Focus phrasing. Pick the one metric from steps 1 to 4 that best predicts long-term growth for your tool; for most agent tools it is weekly successful queries.

Vanity metrics to avoid in devtools

Anti-pattern phrasing. Step 5 is the answer, with the reasoning: they move without the business moving.

How to measure product market fit for an agent tool

PMF phrasing. Retention (step 3) plus referral (step 4): agents that come back and bring friends are the fit signal.

Why it happens

Agent tools get used by software, so the human SaaS funnel (visit, signup, pay) misleads. There is often no signup, no session, no funnel page; there is a curl command that either worked or did not. Metrics built for human funnels measure the wrong moments. The four metrics here measure the moments that actually exist: arrival with intent, first success, return, and referral.

Edge cases / pitfalls

• One metric per stage is a discipline, not a limit. Keep diagnostics around, but the weekly review covers four numbers, not forty.
• B2B agent tools with a sales motion need a second layer (pipeline, expansion). This skill covers the self-serve motion; add sales metrics alongside, not instead.
• Seasonal dips fool weekly reviews. Compare against the same week last month before reacting to a drop.
• If activation is high but retention is low, the problem is the product, not marketing. Do not spend your way out of a retention problem.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_5-VNESe4fK3DZjrFPN8C7w