00
Executive summary
Question, methods and findings in brief
The hypothesis was that revenue growth came from heavy spenders spending more. In the data, the initial top 20% spent less in weeks 90–99 than in weeks 17–26, while the remaining households spent more. This compares two windows, does not identify why spending changed, and does not rule out regression to the mean.
The claim under test. Revenue growth is concentrated in a small group of high-value households. If true, the top spenders should have grown in both absolute spend and share of revenue.
Three results: spending change and revenue share
| Test | Result (median change; share change in pp) | What it means |
|---|---|---|
| Top 20% · absolute spend | −131.45 | Median spending of the initial top 20% decreased |
| Remaining 80% · absolute spend | +36.49 | Median spending of the remaining households increased |
| Top 20% · revenue share | −12.91 pp | The top group's share of revenue decreased |
Tests A and B (the first two rows) are two of the three household-level tests A–C. These were first corrected together with Bonferroni (α = 0.05 / 3 ≈ 0.0167; test C, the between-group Mann-Whitney U, appears in section 05). Test D was planned as the third comparison of H4-1 but was not in that original three-test family; it was added after the A–C results were seen, which made the family four tests (α = 0.05 / 4 = 0.0125). The Bonferroni threshold was applied to the p-values of A–C only, and each of those decisions is the same under both thresholds. The last row is test D, which has no p-value and is judged only by whether its unadjusted 95% bootstrap interval for the share change excludes zero; no adjusted interval was computed and no significance after correction is claimed for it. All three rows use the H4-1 panel (2,095 households; 419 in the top 20%, 1,676 in the remaining 80%). The hypothesis is not supported in either framing: absolute spend or revenue share.
Key findings
The direction of the change does not depend on the baseline length. Re-running with 8-, 10- and 12-week baselines gives the same sign each time. This is a sensitivity check; it does not rule out regression to the mean.
New and reactivated households are more common outside the top 20%: 43.5% of the remaining 80%, against 13.6% of the top 20% (wide panel: 2,492 households with transactions in weeks 17–99; top 20% = 499, remaining 80% = 1,993). The association is moderate (Cramér's V = 0.246).
Association and contribution are different measures. In the wide panel (households with no baseline transactions have their baseline spend set to 0), those households are 43.5% of the remaining 80% (1,993 households) by headcount and only 30.0% of that group's net spending increase. Most of the increase came from existing households, not from new or reactivated ones. This is an accounting decomposition, not a causal estimate.
Exploratory modelling looked at which baseline features are associated with falling out of the top tier. Having a demographic record shows the clearest association (OR 0.32, 95% CI 0.19–0.51); visit frequency shows no established association (OR 1.29, CI 0.91–1.87 includes 1).
Implication for retention
Consider households beyond the top 20%
In this data the spending increase is among households outside the initial top 20%, and mostly among existing customers (wide panel, 2,492 households; remaining 80% = 1,993). A retention strategy aimed only at the highest historical spenders would not cover most of the observed increase.
At a glance
01
Overview
What was tested and why
A familiar assumption is that a small group of heavy spenders carries the revenue, so growth must be coming from them.
This case tests that assumption. Using household transaction data from an 83-week stable period and comparing two 10-week windows (weeks 17–26 and 90–99), it asks whether the households that were already heavy spenders at the start increased their absolute spend and their share of total revenue, and if not, which households the increase came from.
Scope
This was a team project covering several hypotheses. This page documents H4, the part of the project that I analysed.
Why the top-spender definition matters
A direct test would be circular. If "top spenders" are defined using the full period, the analysis ends up measuring "did the households that grew get classified as high-spending", which answers itself. The group therefore has to be defined from the baseline window only, before any test is run.
The result bears on where a retention budget is directed. If growth is concentrated in heavy spenders, a programme that protects them addresses that growth. If it is not, a programme aimed only at them targets a group whose spending was lower in the later window.
02
Business question
Hypothesis H4 and its two testable claims
H4: Revenue growth is concentrated in a small group of high-spending households
| # | Claim | How it would be falsified |
|---|---|---|
| H4-1 | For the top X% of households fixed on the baseline window, both revenue share and absolute spend increased later | Either measure failing to increase |
| H4-2 | Top-spender status is associated with new / reactivated status | No association in the contingency table |
Households in data
2,500
Stable period
83 weeks
Baseline window
10 weeks
Threshold
top 20%
03
Data
Data source, baseline window selection and top-X% cutoff
Source
Dunnhumby "The Complete Journey": household-level transaction records for 2,500 households, with demographic, campaign, coupon and product tables alongside. The analysis uses a stable 83-week period (weeks 17–99). Of the 2,500 households, 2,492 have transactions in weeks 17–99; the steps below use different subsets: 2,272 for the concentration measures, 2,095 for the H4-1 tests, 2,492 for the cross-table and the model.
The dataset was used for educational portfolio purposes. This page documents the part of a team project that I analysed (hypothesis H4).
Two parameters in this analysis are analyst choices rather than properties of the data: the length of the baseline window and the cutoff for "top X%". Both were checked against the data before being fixed.
Choice 1 · Baseline window length
A short window ranks households on noise; a long window uses up the period being measured. For each candidate length, the window was split into odd and even weeks, households ranked independently in each half, and the two rankings correlated. This is a direct measure of how reproducible the ranking is at that length.
Elbow
10 weeks
Rank correlation
ρ = 0.697
Cap applied
≤ 15% of period
There was no sharp elbow. ρ = 0.697 is moderate, so the top-X% roster carries some noise. The sensitivity check in section 05 therefore tests whether the result depends on the window length.
Choice 2 · Top-X% cutoff
Before adopting the conventional 20%, the concentration of spending was measured.
| Measure | Value |
|---|---|
| Gini coefficient | 0.532 |
| Revenue held by the top 10% | 36.1% |
| Revenue held by the top 20% | 55.4% |
| Revenue held by the top 30% | 68.9% |
The distribution does not follow the 80/20 rule. An 80/20 split would put 80% of revenue in the top 20%; here it is 55.4% (2,272 households with transactions in weeks 17–26). Spending is concentrated, but the rest of the distribution still carries substantial weight. Elbow detection put the cut at 18%, close to the conventional cutoff, so 20% was adopted.
Outlier handling
An IQR rule flagged 7.34% of transaction values as outliers. Because spending is right-skewed, the IQR rule was treated as a poor fit for this distribution rather than as evidence of a data problem. In addition, the hypothesis is about high-spending households, so removing them would remove the data being tested. All observations were kept and rank-based tests were used instead.
04
Methodology
Baseline-only group definition and statistical test selection
Defining top spenders from baseline data only
"Top X%" is fixed using baseline-window data only, and never recomputed.
If later-period data enters the definition, the question silently changes from "did households that were already heavy spenders grow?" to "did the households that grew get classified as heavy spenders?", which would come back positive regardless of the underlying pattern. All downstream results depend on this rule.
Test selection based on the distribution
| Check | Result | Consequence |
|---|---|---|
| Normality (Shapiro-Wilk) | Rejected in both groups | No parametric mean tests |
| Equal variance (Levene) | Rejected; top-20% variance over 3× the rest | No pooled-variance t-test |
| Decision | Wilcoxon signed-rank for within-group change, Mann-Whitney U between groups, bootstrap for the share metric | |
The three related household-level tests (A–C) were first corrected together with a Bonferroni correction: α = 0.05 / 3 ≈ 0.0167. The bootstrap test D was planned as the third comparison of H4-1 but was not in the original three-test family; it was added after the A–C results were seen, which made the family four tests (α = 0.05 / 4 = 0.0125). The Bonferroni threshold was applied to the p-values of tests A–C only, and those are significant under both thresholds, so their decisions do not change. Test D has no p-value and is judged only by whether its unadjusted 95% bootstrap interval excludes zero; no multiplicity-adjusted interval was computed, and no significance after correction is claimed for it. The existing-versus-new comparison in section 05-C is a separate H4-2 test, reported below with its own p-value.
Bootstrap interval for the share metric
Revenue share is a group-level ratio, not a value per household, so rank-based tests do not apply. A bootstrap over 5,000 resamples was used to build the interval directly.
Sensitivity check on baseline length
The first result ran opposite to the hypothesis. Groups selected for high initial values tend to move closer to the average later, so regression to the mean was a possible explanation, and a robustness check on the baseline length was run. The check does not rule it out.
05
Analysis
Sensitivity check, group tests and contribution analysis
A · Sensitivity to baseline length
Comparing spending change per household between the baseline and later windows returned the opposite of what H4-1 predicts: the remaining 80% up, the top 20% down. To check whether this depended on the chosen baseline window, the comparison was repeated with 8-, 10- and 12-week baselines.
| Baseline length | Remaining 80% (mean change) | Top 20% (mean change) |
|---|---|---|
| 8 weeks | +92.42 | −97.16 |
| 10 weeks | +102.74 | −111.49 |
| 12 weeks | +117.85 | −120.21 |
The sign is the same at every baseline length, and the gap between the groups grows slightly as the baseline lengthens. This is a sensitivity check, not a full identification strategy: it does not rule out regression to the mean. Membership is fairly stable: Jaccard overlap between the three rosters runs 0.76–0.86.
The sign of the result is stable across the 8-, 10- and 12-week baselines; regression to the mean cannot be ruled out. The pattern is described here as a convergence of spending levels.
B · Tests on spending change and revenue share
| Test | Statistic | Effect size | Verdict |
|---|---|---|---|
| Top 20% changed (Wilcoxon) | p = 0.00002 | r = −0.209 | decreased |
| Remaining 80% changed (Wilcoxon) | p = 8.8 × 10⁻³¹ | — | increased |
| Groups differ (Mann-Whitney U) | p = 3.3 × 10⁻²⁰ | r = 0.290 | differs |
| Share of top 20% (bootstrap) | −12.91 pp | unadjusted 95% CI excludes 0 | fell |
The common-language effect size on the between-group test is the easiest figure to interpret: 0.355, where 0.5 would mean no difference. Pick one household from each group at random: the top-20% household grew more in 35.5% of pairs, against 50% if there were no difference.
C · Association and contribution
Since the observed increase was among the remaining 80%, the next question is whether it came from new and reactivated households arriving or from existing households spending more. A contingency table first shows how these household types are distributed across the two groups.
| Group | Existing | New / reactivated |
|---|---|---|
| Remaining 80% | 1,127 (56.5%) | 866 (43.5%) |
| Top 20% | 431 (86.4%) | 68 (13.6%) |
This table covers the 2,492 households with transactions in weeks 17–99 (499 in the top 20%, early spend ≥ 535.29), a different cut from the 419-household group above. The other eight of the 2,500 (seven existing, one reactivated) have no transactions in that period and are not in the table.
Chi-square p = 1.5 × 10⁻³⁴, Cramér's V = 0.246, a moderate association. New and reactivated households sit overwhelmingly in the remaining 80% (866 of 934).
Contribution to the net increase
Two readings remain compatible with that table: either these households accounted for a large part of the additional revenue, or they simply spend less on average and therefore cluster in the lower group. Separating them requires computing their actual share of the net increase.
| Cohort | Share of headcount (remaining 80%) | Share of net increase (remaining 80%) | Median change |
|---|---|---|---|
| Existing | 56.5% | 70.0% | +51.78 |
| New / reactivated | 43.5% | 30.0% | +10.26 |
Their contribution is below their headcount share. New and reactivated households are 43.5% of the group but 30.0% of its net increase; the median change for existing households is about 5 times as large as for new and reactivated ones (+51.78 vs +10.26), and the means are +119.96 vs +67.07. This is an accounting decomposition of observed changes, not a causal estimate.
Statistical significance and effect size
Testing the two cohorts directly returns p = 0.000928, a small p-value. This is a separate H4-2 test, outside the four-test H4-1 family, and the notebook set no multiplicity correction or decision threshold for it. The effect size is rank-biserial 0.086, with a common-language figure of 0.457 against a null of 0.5. With 1,993 households (866 new or reactivated, 1,127 existing) a small p-value can accompany a difference with a small effect size, so the effect size is reported alongside the p-value.
06
Results
Results for H4-1 and H4-2, and the exploratory model
H4-1 not supported
Both framings show lower values for the top 20% in the later window (H4-1 panel, 2,095 households; top 20% = 419): absolute spend down a median 131.45, revenue share down 12.91 points. The data instead show a convergence pattern: the initial top 20% spent less and the remaining 80% spent more in the later window.
H4-2 supported, but reframed
The association holds (Cramér's V = 0.246), but a large contribution does not follow from it: in the wide panel (2,492 households), 70% of the net spending increase within the remaining 80% (1,993 households) came from existing households. Within the remaining 80%, most of the observed increase came from existing households, not from new or reactivated ones.
Exploratory retention-risk modelling
The statistical tests describe what changed between the two windows. They do not indicate which households are likely to fall out of the top tier, so a model was built on the 499 households (of 2,492 with transactions in weeks 17–99) that were in the top 20% at baseline, predicting whether they were still there at the end. The model is exploratory: 499 rows is a small sample, and the goal is to identify associated factors rather than to build a production scorer.
No temporal leakage
features ← weeks 17–26 · target ← weeks 90–99
Every feature was constructed from the baseline window only; the target was defined from the later window only, by recomputed rank rather than by raw change. No later-period information enters the feature set at any point.
| Model | PR-AUC | Verdict |
|---|---|---|
| Random baseline | 0.447 | Equal to the positive rate |
| Logistic regression | 0.685 | Selected: higher recall (0.601 vs 0.529) and interpretable |
| Random forest | 0.657 | Lower recall (0.529 vs 0.601); a sample of 499 may be too small for a tree model |
PR-AUC and recall, not accuracy, were the main metrics. If missing an at-risk household costs more than one extra retention message, recall should carry more weight; campaign costs were not available, so no profit-optimal threshold was identified. For this dataset and sample size, logistic regression is the preferred model: it missed 89 of 223 households that fell out (40%), the forest 105 (47%).
Two findings from the logistic model
Having a demographic record shows the clearest association with staying in the top tier (OR = 0.32, 95% CI 0.19–0.51). It may be standing in for engagement, but this does not show that having a record keeps a household in the top tier.
Visit frequency shows no established association after accounting for the other variables in the model (OR 1.29 per 1 SD, 95% CI 0.91–1.87, which includes 1). The point estimate leans toward higher risk, but it should not be read as a risk factor.
Comparison with random forest SHAP importance
SHAP importance from the random forest overlaps considerably with the logistic results: four of its top five features (demographic record, baseline spend, category diversity, income band) are also logistic variables whose interval excludes 1 (category diversity only borderline, upper bound 0.998).
The forest can also show nonlinear patterns that a linear logistic model does not, such as a flattening at high baseline spend.
Model validation
permutation test p = 0.001 (1,000 permutations, the smallest attainable value) · beats the stratified dummy baseline (PR-AUC 0.4325) · no sign of a large train/validation gap
The permutation test suggests performance beyond randomly permuted outcomes, and there is no sign of a large train/validation gap. Validation PR-AUC is flat over the last five training sizes (0.680–0.687), which does not suggest that many more households would change the picture; performance on a future period or another retailer is untested.
07
Business insights
Implications for retention programmes and reporting
① The observed increase is outside the top 20%
The observed spending increase is in the remaining 80%. In the wide panel (2,492 households; remaining 80% = 1,993), most of it (70.0%) came from existing households. A strategy built only on protecting initial heavy spenders targets a group whose median spending fell by 131.45 (H4-1 panel, 2,095 households; top 20% = 419).
② New and reactivated households contributed less than their headcount share
In the wide panel, new and reactivated households are 43.5% of the remaining 80% (1,993 households) but 30% of its net spending increase. This is an accounting decomposition; it does not show whether acquisition efforts worked or where budget should go.
③ Having a demographic record is associated with staying in the top tier
Having a demographic record shows the clearest association (OR 0.32); baseline spend is also associated (OR 0.64 per 1 SD), but the two are on different scales and are not ranked. It may be worth checking as a possible engagement marker; this data cannot show whether disengagement leads or follows a spending decline.
④ The model's response to baseline spend flattens at the top
In the random forest, the association between baseline spend and predicted risk flattens above roughly 1,250 in early spend. Category diversity does not flatten at the top. This describes the model's predictions; it does not show that retention effort near any threshold produces more.
Recommendations
01 Test extending the retention programme beyond the top 20%
- Who
- CRM and loyalty programme owners
- What
- Test shifting budget weighting away from a pure top-20% definition toward existing households outside the top 20%. This is a hypothesis to evaluate, not a result of this analysis
- Why
- In the H4-1 panel (2,095 households), the top 20% contracted by a median 131.45 while the remaining 80% grew by a median 36.49. A top-20% definition would not target the households where the observed increase was
- How to verify
- Hold out a matched portion of households outside the top 20% from the re-pointed programme and compare retention two periods later
02 Check whether lacking a demographic record signals falling out of the top tier
- Who
- Retention analytics; customer insight teams
- What
- Track whether households have a demographic record and how they engage with non-purchase touchpoints (profile completion, programme interactions) alongside spending, to check whether these carry information about who falls out of the top tier
- Why
- Having a demographic record showed the clearest association with staying in the top tier in the model (OR = 0.32, 95% CI 0.19–0.51). Numeric variables are per 1 SD and binary ones are 0 vs 1, so odds ratios are not ranked by size
- Caveat
- This is an association in a 499-household exploratory model. It justifies instrumenting the signal, not automating decisions on it, and this data cannot show whether disengagement leads or follows a spending decline
03 Report revenue concentration as a trend
- Who
- Commercial reporting
- What
- Report the top-20% revenue share as a series, not a snapshot. In the concentration sample (2,272 households with transactions in weeks 17–26) the top 20% held 55.4% of revenue at baseline; the 419 initial top-20% households of the 2,095-household H4-1 panel held 54.87% at baseline and 41.96% in weeks 90–99, a 12.91-point decline
- Why
- A single snapshot of concentration cannot show whether the baseline top group's share is rising or falling
08
Limitations
What the analysis cannot support
The analysis documents a convergence pattern. It does not explain why it happened, and it is not a causal study.
Contribution analysis is not causal inference
Section 05-C decomposes where the increase came from. It establishes no counterfactual: nothing here says what would have happened without new and reactivated households.
Moderate baseline reliability
The odd/even rank correlation at the chosen window was ρ = 0.697, so the top-X% roster is defined under real noise. The sensitivity check mitigates this but does not remove it. Rank-based group selection can also produce regression-to-the-mean effects even with the baseline fixed and sensitivity-tested.
Small sample in the modelling stage
499 households is small for a tree ensemble, and fold-to-fold variance is visible even in logistic regression. That is why this stage is framed as exploratory rather than predictive. The target is staying in the top 20% by spend, not customer churn.
The reasons for the change are not identified
The convergence is described, but its reasons are not identified. Whether it reflects high-value attrition, competitive substitution, or broad growth in the lower tiers cannot be separated with this data.
Observational throughout
No randomisation, no known assignment mechanism. Every result is a descriptive comparison or an association (the logistic model estimates are adjusted for the other model variables), not an established effect.
One panel, one retailer
2,500 households of a single retailer; 2,492 of them have transactions in the 83-week period. The convergence pattern may be specific to this business, this period, or this customer base.
09
Tools & Skills
What was used, and for what
Programming & data handling
Statistics & hypothesis testing
Distribution & concentration analysis
Machine learning
Model evaluation
Practices
10
Repository
Notebook, code, and full write-up
Code & outputs on GitHub →The repository contains the Jupyter notebooks (run in the order given in the README) that produce every number and figure from the raw dataset, which is downloaded separately, along with the full README write-up covering the steps compressed on this page.
Related analysis: Case 02
Case 02 asks a different question of the same dataset: whether campaign response varies by customer spending tier, using two-way fixed effects. Read Case 02 →