Vanity reports: 5 questions to find out if your agency measures what matters
June 17, 2026
A vanity metric is any number that rises without the business getting better: impressions, reach, followers, likes. They measure activity, not results. The metrics an owner needs are four — how many leads came in, what each one cost, how many became customers, and how much they billed — and a report that does not answer them is decoration.
Every month the agency report arrives: reach going up, impressions in the thousands, a likes chart with a green arrow. And at the end of the month, the question the report never answers: how many customers did this bring?
Vanity metrics vs. business metrics
A vanity metric is any number that rises without your business getting better. Impressions, reach, followers, likes — they measure activity, not results. They are not useless as intermediate signals; they are useless as a conclusion.
Business metrics answer owner questions: how many leads came in, how many became customers, what each one cost, how much they billed. They are harder to measure — and that is exactly why vanity reports exist: they are the easy path.
The 5 questions
At the next monthly meeting, ask these. The quality of the answers tells you everything.
1. “How many leads did we generate this month, and from which channels?” If the answer mixes “engagements” with leads, there is a problem. A lead is an identifiable contact with demonstrated interest — a name and a phone number, not a click.
2. “What did each lead cost, by channel?” Total spend divided by leads, channel by channel. If Google costs €12 per lead and Instagram €80, that decides where the next euro goes. Anyone who does not know this number is allocating budget on intuition.
3. “How many of those leads became customers?” The question that separates agencies from systems. It requires someone to connect marketing to your sales record — CRM, diary, invoicing. If nobody built that connection, nobody knows whether the marketing works. Full stop.
4. “What did the tracking miss?” A technical question with a revealing answer. Between ad blockers and iOS restrictions, browser-only tracking loses a meaningful share of conversions. Anyone measuring seriously knows their blind spots and compensates for them (server-side tracking); anyone who is not says “the numbers are all in Analytics”.
5. “What are we cutting this month?” Real measurement produces decisions — and the hardest decision is cutting. If twelve months of reports never cut anything, nobody is optimising; somebody is renewing a retainer.
The final test
A useful report reads in five minutes and ends with three decisions. A vanity report reads in thirty seconds of scrolling and ends with “continue monitoring”.
If the five questions went unanswered, the problem may not even be the agency — it may be the measurement infrastructure nobody ever built. But it is yours: it is your budget deciding blind.