Back to the journal
Business & work 4 min read

Did That Website Change Help? Read a Small Sample Carefully

Compare website enquiry counts and rates without overstating a small sample. Keep denominators, missing data and other changes visible.

A clearly synthetic comparison shows 4 completions out of 80 eligible sessions before and 6 out of 120 after. Both rates are 5%, despite different raw completion counts; no causal claim is made.
The short version

Keep raw counts, denominators, missing observations and competing changes beside any rate comparison; do not turn a small sample into a causal claim.

Your website received four enquiries in one period and six in the next. It is tempting to announce that the new page produced fifty per cent more leads. First ask how many eligible visits each period contained, what counted as an enquiry and what else changed.

A small before-and-after comparison can still be useful. It can show what you observed, expose a broken measurement and identify the next question. It rarely supports the confident causal story suggested by two numbers alone.

Use the fictional counts below to make a careful comparison. They are invented for arithmetic practice and do not describe AI Empower or a client.

Put counts beside rates

In Period A, there are 80 eligible visits and four visits with a completed enquiry. In Period B, there are 120 eligible visits and six visits with a completed enquiry. For this exercise, an eligible visit is a recorded session reaching the relevant service page, and a completed enquiry means a confirmed accepted form submission during that session. Count each session at most once in each numerator.

ObservationPeriod APeriod B
Eligible recorded sessions80120
Sessions with a confirmed enquiry46
Observed completion rate5%5%

The enquiry count rose by two while the recorded session count rose by 40. Both rates are five per cent. These arithmetic facts do not tell you whether the page caused a change in visitor behaviour. They also do not tell you whether the enquiries were suitable for the business.

Now change only Period B’s denominator to 100. Six divided by 100 gives six per cent. Compared with five per cent, that is a one-percentage-point difference, or a twenty per cent relative increase in the observed rate. Keep the count of six beside either description. A relative percentage alone can make a small difference sound much larger than the evidence warrants.

Make sure both periods mean the same thing

A tracking event called “lead” can represent different actions in different implementations. Did it fire on a button click, a form-validation attempt, a successful server response or a later staff review? If the trigger changed, the periods may measure different things.

Google Analytics’ recommended events distinguish a submitted lead from later lead qualification and conversion. Those names are useful reference points; they do not verify your installation. Have the responsible person inspect when your event fires and how duplicates, spam and test submissions are handled.

Use matching units. Do not divide a count of enquiries across email, phone and forms by sessions on only the form page and call the result a website completion rate. Likewise, a visitor-level numerator needs a compatible visitor-level denominator. If the records cannot be reconciled, report the counts separately and explain the gap.

Record the changes that compete with your explanation

Suppose Period B includes a newsletter linking to the service page. Its readers already know the business. Or a public holiday changes the kinds of enquiries people send. A campaign, a change in opening hours, a new offer or a tracking interruption can all complicate the comparison.

You do not need to invent a correction factor. Add an observation note:

“During Period B, a newsletter linked directly to this page. We cannot separate that traffic change from the page revision in this comparison.”

A before-and-after view leaves those explanations mixed together. A properly designed controlled experiment may answer a narrower causal question, but it requires suitable traffic, design and analysis. This worksheet does not supply a significance test or choose a winner.

Try the evidence panel

Create two columns and fill in the following for each period:

  1. Start and end dates, time zone and why the windows were chosen
  2. The exact page version and visitor task
  3. Eligible audience and counting unit
  4. Completed outcome definition and event evidence
  5. Raw numerator and denominator, including exclusions
  6. Known gaps, tracking changes and other business changes
  7. A sentence stating only what the records support

Treat zero and missing differently. Zero confirmed enquiries from 20 measured eligible sessions yields an observed zero per cent. An unavailable enquiry count yields an unknown rate. A denominator of zero also gives no usable rate; do not display zero per cent as if the business had observed ordinary traffic with no completions.

Week 32 · interactive local candidate

Keep counts, rates and uncertainty together

Fictional local exercise. This exercise keeps its working inputs in page memory and starts over on reset or reload. It does not automatically submit or save those inputs, call an AI or access accounts. Copies and printouts are outside reset; browser-managed history, extensions and device behaviour are outside this exercise’s control. Use invented examples only; do not enter personal records or credentials. No real transfers or independent verification are performed.

The starting example has four enquiries from 80 sessions and six from 120. Change Period B to 100 sessions, then compare the count difference, percentage-point difference and relative rate change. Blank counts remain unknown; zero is a distinct observation. Editing period context or evidence resets comparability to Not established, so matching definitions must be reconsidered.

Period A
Period B

Review pending. Change inputs, then review the current version. Earlier results are cleared after every edit.

Use the text-only exercise

The complete unchanged manuscript remains readable if the controls are unavailable.

Return to Try the evidence panel

Write the conclusion before choosing the next action

For the first fictional example, a defensible note is: “The recorded enquiry count increased from four to six. The observed session completion rate stayed at five per cent. The periods contained different numbers of eligible sessions, and this comparison does not establish an effect of the page change.”

Then choose an action matched to the uncertainty. Repair inconsistent tracking before collecting more of the same data. Investigate a broken completion path immediately. If the page remains usable and the evidence is simply sparse, keep the result labelled inconclusive and decide what additional observations would help.

Use the organisation’s approved analytics and consent arrangements. The exercise requires aggregate counts, not names or enquiry contents. Your deliverable is a small evidence panel and a careful decision note, not a performance promise.

Review your website’s enquiry journey with AI Empower.

Primary sources checked 4 October 2026

Sources & review

Primary sources checked on . The checklists and planning examples are AI-assisted editorial guidance, not source quotations or reported client results.

This AI-assisted guide uses fictional examples for practice. It does not report client results or establish that a live system will behave the same way.

Originally published: .