Digital product analytics: the practical version

What to take away

A digital product seller has three questions that matter more than any chart: did the buyer get the thing they paid for, did it work when they opened it, and did they come back for the next one. Everything below serves those three.

Start from the value event, not the dashboard

A value event is the moment the product does its job. For a course platform it is a completed module with the exercise submitted. For a template pack it is a file opened and edited. For a membership it is a member using a benefit that is not the welcome email.

Pair every value event with an output check. A downloaded ebook is not a read ebook. A generated invoice is not a valid invoice until the accounting software accepts it. A subscription is not delivered until the entitlement actually grants access.

The pairing matters because the two fail separately. Delivery can succeed while the output is wrong, which shows up later as a refund. Track both, and track the correction that followed.

Build the metric tree from outcomes down to inputs

Put one durable outcome at the top. For most digital products that is repeat purchase or renewal. Then list the conditions that feed it: a buyer who fits the product, a first value event inside the trial or refund window, delivery that holds, and a price the buyer expects to pay again.

Each branch gets one measure and one owner. A launch team might own first value within seven days while guarding refund rate, checkout errors and support contacts. When a branch has no owner, it has no decision, and the metric is decoration.

Keep the tree honest about direction. A branch shows a plausible relationship, not a proven cause. Renewal and support volume move together for reasons that have nothing to do with each other.

Write a metric dictionary with versions

Version the definition rather than editing it. When a platform changes how it counts a session or a refund, annotate the date and decide whether to restate history or mark a break in the series. A chart with an unmarked break is worse than no chart.

Store the query next to the definition. If the number cannot be reproduced from source data by someone else, it is an opinion with a decimal point.

Model entities before you name events

Map the things first: user, account, workspace, plan.

Then name events against that model. Each event records several details. These are who acted, what object was involved, and the result. They also include the time, the source, and the schema version. Separate an attempt from a success. A checkout attempt that fails at the card step is a different fact from a completed order.

Keep changing values out of event names. A plan tier or a price belongs in a parameter, not in an event called upgraded_to_pro_annual_49.

Collect only what a decision justifies

Keep a data-use register. Each field gets a purpose, an access role, a retention period, a deletion behaviour and an owner. Curiosity is not a purpose, and a field collected for a test that ended two years ago is a liability.

Free text, uploaded documents, full URLs and location deserve review before collection, not after. Consent choices and platform privacy settings change who you can see, so report coverage alongside every result. A conversion rate computed on consented traffic only is a rate for consented traffic.

The W3C Privacy Principles statement gives designers shared language for this and warns against pushing privacy work onto the individual. It is not legal advice, and it does not replace a review of your own configuration or your contracts.

Test the data before you trust a number

Run the list after every release that touches checkout, entitlement or the event schema. Most broken dashboards are broken by a deploy, not by a slow query.

Reconcile behavior against money

A purchase event is not revenue. Revenue is what the processor settled, minus refunds and chargebacks, net of tax you collected on someone else's behalf. Reconcile the two monthly and investigate any gap above your own tolerance.

Tax treatment of digital goods varies by state and by whether you sell a download, a subscription or a service. Some states tax digital products, some do not, and marketplace facilitator rules can move the obligation to the platform.

Check your state revenue department, or the CDTFA if you sell into California, and confirm with a CPA before you set a price.

Support demand belongs in the same segmentation as adoption. A feature that lifts engagement while tripling tickets has not obviously paid for itself. Read the tickets behind the aggregate before you call it a win.

Judge a launch on mature cohorts

Group buyers by a starting condition: launch week, acquisition source, plan, or first value. Compare cohorts at the same age. A cohort that is three weeks old cannot have a fair 90-day refund rate, and a launch report written at day seven will flatter every product.

Choose retention logic that matches natural use. A monthly compliance tool is not judged by daily logins. Show the cohort definition, the starting event, the return event and the incomplete periods, then let the reader see the shape.

The same discipline applies when a digital product launch is reviewed after the fact, and it is the reason launch benchmarks from other companies rarely transfer.

Use experiments only for causal questions

Write the hypothesis, the eligible population, the assignment unit, the change, and the main outcome. Also write the guardrails and the stop rule before you start. Check allocation, contamination, novelty effects and concurrent releases.

A low p-value does not tell you whether the change is worth shipping. Effect size does, and so does the cost of the guardrail you traded away.

Some questions never need a test. A broken checkout, a false claim or a security gap gets fixed. Interrupted time series, staged rollouts and matched comparisons cover the operational cases where randomisation is not available or not ethical.

Set a review cadence

Open every review with data health. A retention number from a pipeline that dropped events for two days is not a retention number.

Close every review with a written record: what changed, what did not, who decided, and when the conclusion gets revisited. Retire metrics that no longer serve a decision.

Where the method meets the product

A product decision about pricing, packaging or onboarding should name the metric it expects to move and the guardrail it refuses to break. If it names neither, the decision is a preference.

A membership site marketing review asks the same question this article asks of every metric: which member behaviour counts as value, and who decided that.

The GAO evaluation design guide sets out how evaluation questions, evidence needs and design choices fit together. It is written for federal programme evaluation, so use its design discipline as a check on method rather than as proof that a marketing result caused an outcome.

The GOV.UK technology selection guidance recommends choices that can change over time, keep control of data, address security risk and account for ownership cost. Those are useful buying questions for an analytics stack, though they are public-service rules rather than private-sector mandates.

The GOV.UK Service Manual also warns against relying on digital analytics alone when measuring service success, and suggests pairing performance measures with research, feedback and financial data. A commercial product has different obligations, but the evidence principle carries over.

Common questions

What is the first metric a digital product should define?

Define the value event for one product and one audience, then pair it with an output check and a refund measure. That trio answers whether the product worked before you look at anything else.

Is product analytics the same as web analytics?

No. Web behaviour contributes, but product analytics also covers entities, entitlements, delivery quality, support and the money records. A pageview count cannot tell you whether a buyer received a working file.

How many metrics should a team track?

As few as the current decisions require: one primary outcome and enough guardrails to expose the tradeoffs. Every extra metric needs an owner, or it should be retired.

Can analytics prove why buyers behave a certain way?

Rarely on its own. Use research or a causal design, state the assumptions, and keep the remaining uncertainty visible in the write-up.