Click for a Printer-friendly Version
- Adobe PDF
The Superiority of Statistics-Based Predictive Models Versus
By Jim Wheaton
Principal, Wheaton Group
Original version of an article that appeared in "The
DMA's 2001 Research Council Journal"
For those of us who live in the Southeastern United States, "Kudzu"
is a four-letter word. A plant native to Japan, it grows like
crazy and — the point of this analogy — is difficult
The same is true of Recency-Frequency-Monetary ("RFM")
Cells, which have thrived for years despite the existence of more
sophisticated statistics-based predictive models. RFM is a
1970's approach that is ill suited to today's centralized, atomic-level
data repositories. Time-and-time-again, carefully constructed
and implemented predictive models have proven to be superior to
RFM Cells. This is true on an absolute as well as a cost-versus-benefit
Experts argue about which modeling technique is superior —
regression, neural networks, genetic algorithms, and the like.
But they generally agree that RFM should be relegated to history's
dustbin, to paraphrase a famous nineteenth century analyst.
In fact, there exist only two possible end results of RFM:
- A stable and easy-to-implement, but crude, segmentation strategy.
- A complicated, sometimes sophisticated approach that is difficult
to implement and often is unstable.
This article is divided into three sections:
- The first will compare predictive models with RFM Cells, in order
to develop an understanding of their similarities and differences.
- The second will detail a financial simulation of a single promotion
by a hypothetical direct marketer with 300,000 active customers,
using conservative assumptions, to understand further the dynamics
behind the cost-effective superiority of predictive models —
even for companies with modestly sized customer files.
- The third will outline the improvements that were gained in circulation
efficiency by two direct marketing companies that replaced RFM Cells
with statistics-based predictive models.
RFM Cells Versus Statistics-Based Predictive Models
The Limitations of RFM Cells
Consider a relatively large
direct marketer that uses the following characteristics to classify
its one million active customers:
- Recency, or the number of months since the most recent paid
order. Each customer is placed in one of four possible monthly
categories: 0-6, 7-12, 13-24 or 25+ months since the last
- Frequency, or the total number of historical orders. Individuals
are classified by 1, 2 or 3+ previous orders.
- Monetary, or historical Average Order Size. Three equally-sized
customer groups are created: "Low," "Mid,"
The result is 36 (i.e., 4 x 3 x 3) Recency/Frequency/Monetary
permutations, or RFM Cells. With an average cell quantity
of 27,778 (i.e., one million / 36), sufficient sample size is available
to analyze past promotions and construct a stable selection hierarchy.
Also, the 36 cells are easy to implement.
Let's examine how these 36 RFM Cells are used to determine whom
to contact. First, some assumptions:
- The direct marketer is willing to promote to the breakeven of $1.25
per piece mailed.
- A re-mail is done several weeks after the initial drop, with performance
that is 50% of the initial contact.
To determine which customers
to promote, historical results are analyzed to develop by-cell estimates
of future purchase volume. Then, a cell hierarchy is created,
from highest to lowest estimated performance. Those cells
that are expected to do at least $1.25 per piece mailed are contacted.
Likewise, those at $2.50 or more on the initial drop are targeted
a second time.
Consider the crudity of such a strategy. Of all the information
contained in the database, only three of the fields are being used.
And, even these three fields are not being optimally leveraged.
Consider, for example, the Frequency input to this 36-cell segmentation
strategy. Customers who have purchased — say —
ten times, have been categorized identically to those who have bought
just three times. In reality, they have been much more loyal
and should be designated as such.
Recognizing this problem, the direct marketer decides to expand
the number of categories for the Recency and Frequency portions
of the RFM Cells:
- For Recency, each customer is placed in one of seven possible monthly
categories: 0-6, 7-12, 13-18, 19-24, 25-36, 37-48, and 49+
months since the last order.
- For Frequency, four groups are created: 1, 2, 3-4, and 5+
This results in 84 (i.e., 7 x 4 x 3) cells.
And, the average number of customers per cell of 11,905 (i.e., one
million / 84) remains sufficiently large to analyze past promotions
and construct a selection hierarchy.
The direct marketer then tries to incorporate additional key database
fields into the selection process, including:
- Merchandise categories.
- Satisfaction indicators, such as returns, exchanges and allowances.
- Service measures, such as backorders and out-of-stocks.
- Payment indicators, such as cash versus credit, and phone versus
- Promotion history.
Consider just the merchandise categories. Our direct marketer
sells hundred of SKU's, with price points ranging from $9.99 to
$995. Clearly, not all of the merchandise is created equal,
and a given customer's past purchase patterns will be a strong predictor
of future behavior.
Although segmentation by SKU is not practical, the direct marketer
decides to create eight groupings. With this, the number of
cells jumps to 672 (i.e., 7 x 4 x 3 x 8), with an average quantity
per cell of just 1,488. This is too small to achieve a consistently
valid read of response performance, even though the customer count
is a relatively large one million. This problem will be exacerbated
for the majority of direct marketers that have significantly smaller
Also, the account manager at the direct marketer's computer service
bureau has to create over 1,000 lines of lines of Boolean logic
each time the RFM strategy is implemented. After all, records
must be selected, test panels created, and mail keys assigned.
This, of course, increases dramatically the opportunity for error.
Clearly, including all of the key database fields into the selection
process will create a system that would have confused Rube Goldberg.
Even if a stable selection hierarchy could be achieved, no service
bureau could guarantee consistently timely and error free execution.
Recognizing the futility of the endeavor, the direct marketer begins
to search for an alternate approach.
Predictive Models — A Selection Tool for Today's Sophisticated
Predictive models do not have these limitations.
All database fields with the potential to isolate future buyers
from non-buyers can be evaluated. There are none of the sample-size
issues that are inherent in RFM Cells. And, the result of
the model — a rank ordering of customers by an individualized
estimate of predicted future purchase behavior — will result
in a straightforward implementation. All customers above a
predetermined predicted performance will be promoted, and the balance
To understand why this is true, it is important to understand how
statistics-based predictive models work. Please refer to Table
1 as we walk through a very simple example of how two hypothetical
customers, Jack and Jill, are evaluated by a model in terms of their
future predicted purchase behavior from a national retail chain:
Predictive Model: A
This model contains the following independent (i.e., predictor)
- The standard Recency, Frequency, and Monetary (i.e., Average Order
- Two supplemental measures of whether or not the following conditions
exist: An order from "Department 7," and Gender
- The distance in miles between each customer's home and the retailer's
A predictive model works by determining each individual's value
for every independent variable, and then multiplying it by a number
called a Coefficient. The Coefficient can have either a positive
or a negative sign. The Product is called a Partial Score.
The Products of each of these multiplications are then summed to
create an overall number called a Score. The higher the Score,
the more favorable the predicted future behavior.
Consider the first independent variable, Recency. Because
the Coefficient has a negative sign, the longer it has been since
the most recent order, the lower the Partial Score, and the more
negative the impact on the overall prediction of future purchase
behavior (i.e., the Score):
- Jack has not ordered for 18 months. This is multiplied by
negative 0.060 to create a Partial Score of —1.080.
Jack, in a sense, starts out "deep in the hole."0*
- Jill, on the other hand, ordered just two months ago. Therefore,
with a Partial Score of negative 0.120, she starts out in a more
favorable position than Jack.
The next variable is Frequency.
Because the Coefficient is positive, the higher the number of previous
orders, the higher the Partial Score, and the more positive the
impact on the overall prediction of future purchase behavior:
Jack has four lifetime orders. This is multiplied by 0.270
to create a Partial Score of 1.080. Now, Jack's Cumulative
Partial Score is 0. His favorable Frequency profile has, in
a sense, cancelled out his unfavorable Recency.
Jill, on the other hand, has just two lifetime orders. Therefore,
with a Cumulative Partial Score of 0.420, she remains "ahead"
Jack falls further behind once his inferior Monetary Partial Score
is added to the equation. However, note that Jack has previously
ordered from Department 7. This merchandise category has a
major positive impact on the overall prediction of future purchase
behavior. For the first time, he is ahead of Jill, with a
Cumulative Partial Score of 2.400 to 1.635.
This so-called jockeying for position continues throughout the remaining
two independent variables, Female and Distance. When it is
all said and done, however, Jack and Jill end up with the same Score
When all of the retailer's customers are rank ordered by Score,
Jack and Jill end up in the identical position. Despite the
fact that they display stark differences on each of the six independent
variables, the two are equivalent in terms of their predicted future
Consider the advantages of this predictive model versus RFM Cells:
- Formal statistical procedures were employed to systematically evaluate
all available potential independent variables, and identify a subset
of six that — together — optimally predict future purchase
- Each customer is evaluated individually on each of these six independent
variables, thereby eliminating the cell proliferation that makes
it difficult to achieve a statistically valid read of response patterns.
- All of the database information that corresponds to each of the
six independent variables is being leveraged. Consider, for
example, Frequency. The higher the number of previous orders,
the higher the Partial Score, and the more positive the impact on
the overall prediction of future purchase behavior (i.e., the Score).
Contrast this to the earlier RFM example, where customers with ten
previous purchases were being categorized identically to those with
- Because each customer has been evaluated individually, promotional
decisions can be made on an individual basis. This is particularly
advantageous when a pre-set quantity is available to be mailed.
Consider a re-mailing where 51,000 pieces are available for the
promotion. Invariably in such circumstances, the required
quantity will "split" one of the cells in the hierarchy.
To counteract this phenomenon, additional Boolean logic will have
to be incorporated into the selection criteria.
- Proper weights (i.e., the Coefficients) are assigned to each independent
variable, thereby improving the accuracy of the predictions.
In short, predictive models are more stable than RFM Cells.
They are easier to implement. And, they do a substantially
better job of determining future purchase behavior.
Many in our industry, while
conceding that statistics-based predictive models are superior to
RFM Cells, believe that they are cost-justified only for large direct
marketers. The following financial simulation will show why
even modestly sized companies should embrace predictive models.
Consider a single promotion by a hypothetical direct marketer with
300,000 active customers. This promotion will be segmented
first by traditional RFM Cells and then by a predictive model.
Even with conservative assumptions, we will see how an investment
of — say — $25,000 in a predictive model will return
an incremental $9,000 in revenue and $33,900 in profit over traditional
RFM Cells after just one promotional cycle; that is, a single mailing
followed several weeks later by a re-mail. At $33,900 thousand
per promotion, it is clear that the profitability gains over a full
year will be profound.
The starting point for our financial simulation is the underlying
promotion assumptions outlined in Table 2:
Organizing these promotion assumptions into the Table 3 worksheet
below shows how mailing all 300,000 customers generates $450,000
in sales but $0 in profit. With nothing left over for critical
functions such as customer acquisition, an indiscriminate promotional
strategy results in a business with no future. What is needed
to ensure long-term viability is some kind of segmentation strategy.
For decades, direct marketers have understood the need for segmentation.
Traditionally, they have resorted to RFM Cells. The idea behind
any segmentation strategy is to identify and eliminate from a given
promotion all individuals whose predicted performance is below breakeven.
With this in mind, we must first establish the breakeven response
rate for our promotion. As outlined in the Table 4 worksheet,
this works out to 1.54%, or 15.4 orders per 1,000 pieces mailed:
The breakeven response rate has been calculated without regard to
company overhead. This is because — arguably —
breakeven is a consideration only at the margin; that is, for those
customers who are borderline candidates for promotion. If
overhead is not comfortably exceeded by the more productive customer
segments, then structural business problems exist that even segmentation
will not cure.
Table 5-A illustrates just how an RFM segmentation strategy works.
Before discussing this in detail, however, some background information
- For the sake of simplicity, only 10 RFM Cells have been created.
In reality, RFM segmentation strategies range from a few simple
cells to ones that number in the thousands.
- For our example, the absolute number of cells is not important.
Instead, the key issue is how well they differentiate those customers
who are likely to respond from those who are not. And, the
measurement of this discriminatory power involves a concept called
- "Lift" in Table 5-A is defined as the ratio of a given
RFM Cell's response rate to the overall rate of 2.00%. For
the top 10% of the file, which is defined by Cell 1, the response
rate of 4.00% translates into a ratio-to-average of 2.00 (i.e.,
4.00% / 2.00%). The bottom 10% or Cell 10, on the other hand,
has a ratio-to-average of 0.40 (i.e., 0.80% / 2.00%).
Although in the real world one never sees RFM Cells that are uniformly
divided into 10% chunks of the overall universe, a "lift"
of 2.00 for the cells that correspond to the approximately top 10%
of customers is common.
Segmentation Strategy #1 — RFM Cells
(1.54% = Breakeven Response Rate)
As is evident in Table 5-A, RFM Cells — as with any other
segmentation strategy — do nothing more than re-sequence the
customer file from most to least likely to respond. This is
why, if all ten RFM Cells were mailed, the total revenue would remain
at $450,000 and the total profit at $0.
Notice that the Contribution to Overhead and Profit for Cells 8,
9 and 10 is negative $1,350, negative $4,275 and negative $7,200,
respectively. This is not surprising, given that their corresponding
response rates of 1.40%, 1.10% and 0.80% are below the 1.54% breakeven.
Although these cells are generating incremental revenue, overall
profitability is lowered in the process. We would be better
off sacrificing this additional revenue for the sake of the bottom
Table 5-B illustrates the effect of mailing only the seven profitable
RFM Cells. Although revenue is down almost $75,000, to $375,750,
we now have a mailing that is $12,825 in the black — or 3.41%
of total sales. Although not outstanding performance, it is
a significant improvement over indiscriminately mailing the entire
RFM Cells (cont.)
(1.54% = Breakeven Response Rate)
Let's now replace the RFM Cells with 10 segments generated by a
statistics-based predictive model. Again, the concept of lift
will be used to illustrate the segmentation power of the model.
As is seen in Table 6-A, Segment 1's response rate of 6.80% has
a ratio-to-average of 3.40 versus the overall response rate of 2.00%.
This level of lift is often seen in models built off databases with
a wealth of detailed and accurate transaction history.
Segmentation Strategy #2 — Predictive Model
(1.54% = Breakeven Response Rate)
Notice also that the bottom segments have much lower response rates
and lifts than their RFM counterparts. Segment 10, for example,
has a response rate of 0.40% and a ratio-to-average of 0.20 versus
RFM Cell 10's 0.80% and 0.40. This is because the predictive
model is doing a much better job of concentrating high-probability
responders in the top segments and low-probability responders in
the bottom segments. (Under ideal circumstances, top-10%-to-average
lifts of over 4.00 are attainable, as are bottom-10%-to-average
lifts of about 0.15.)
Because our predictive model is doing such an efficient job of concentrating
high-probability responders in the top segments, Segments 6 and
7 join Segments 8 through 10 in qualifying for elimination.
As is apparent in Table 6-B, mailing only the five above-breakeven
segments generates $353,250 in revenue, which is even less than
with the RFM strategy. However, profitability is up significantly
to $33,075, or 9.36% of sales. Again, revenue has been sacrificed
Predictive Model (cont.)
(1.54% = Breakeven Response Rate)
Some readers might be concerned with a predictive modeling strategy
that sacrifices revenue for the sake of an improved bottom line.
Fortunately, this effect is counteracted in the form of improved
Re-mailings generally result in a response rate decline, and often
at about 50% compared with the initial mailing. With this
assumption, the re-mail strategy for our hypothetical direct marketer
will be targeted only to segments that are sufficiently responsive
to remain above breakeven with a 50% response rate decline.
As is seen in Tables 7-A and 7-B, only RFM Cell 1 and Predictive
Model Segment 1 meet this criterion. (This is a conservative
assumption because a predictive model often generates a larger number
of profitable re-mail segments than does an RFM approach.)
Predictive Model Segment 1, because of its superior concentration
of high-probability responders, performs much better than RFM Cell
1: $76,500 versus $45,000 in revenue, and $18,150 versus $4,500
Re-Mail Performance — RFM Cells
(1.54% = Breakeven Response Rate)
Re-Mail Performance — Predictive Model
(1.54% = Breakeven Response Rate)
Tables 8-A and 8-B tally the results of the original mailing and
the re-mailing. The predictive model has a small, $9,000 revenue
advantage over the RFM Cells. On the profit side, however,
the model has a substantial, $33,900 advantage. In short,
the model has enhanced profitability significantly while increasing
Overall Revenue, Model Versus RFM Cell
Overall Profit, Model Versus RFM Cells
Summary of Financial Simulation
An investment of —
say — $25,000 in a statistics-based predictive model by a
modestly sized direct marketer will more than pay for itself in
the first promotional cycle alone. Even assuming $1 or $2
per thousand for scoring, there will be an immediate impact on the
bottom line. And, after the first promotion, a modestly sized
direct marketer can look forward to an annuity of $33,900 per promotional
cycle. There are few investments available with such a favorable
For Those Who Insist on a Cell-Based Selection System: A
If RFM is inferior, then why
is it so popular? One reason is fear of the unknown.
Many direct marketers are suspicious of statistics-based predictive
modeling techniques because they don't understand them. If
they have difficulty comprehending what lies behind "Regression
Segments 6 through 10," for example, it will be a challenge
to explain to the CEO why they should not be mailed.
For all of its shortcomings, RFM is understandable. Every
direct marketer can explain why a customer with — say —
just one previous order, five years ago, for $5, should not be contacted.
Fortunately, even direct marketers who insist on an easy-to-understand,
cell-driven segmentation strategy should discard RFM. This
is because there exist statistics-based tree analysis tools that
are far superior.
One such product is "CHAID" (Chi-Square Automatic Interaction
Detection). Another is "CART" (Classification and
Regression Trees). P.C. versions of these products are available,
and can be executed by marketers without an advanced degree in statistics.
Tree analysis is similar to RFM in that each node (i.e., cell) is
defined by easy-to-interpret Boolean statements. The technique
methodically divides universes into multiple groups with response
rates that are significantly different from each other. Table
9 is a very simple hypothetical tree:
From a customer universe with an overall performance of 2.42%, the
tree software determined that Frequency is the primary differentiator
of response. Customers with at least two previous orders at
the time of a promotion did 3.27%. This is an improvement
of 89% versus the 1.73% for those with just one order.
Within the one-order cell, the software identified Recency as the
most important driver of response. Customers whose single
order was within the past six months did 2.45%, a slight improvement
over the entire customer universe of 2.42%. However, the subset
whose order was at least seven months ago responded at just 1.49%.
Assume that a retailer wants to segment its customer file by response
to a previous mailing. To keep matters simple, a tree model
will be built off just six variable types: Months Since Last
Order, Number of Orders, Average Order Size, Merchandise Category,
Age, and Gender. Assume also that the result is 31 nodes,
including the following:
- Female jewelry buyers with four or more purchases, averaging $500+,
at least one purchase within the past six months, and living within
five miles of a store.
- Male electronics buyers with three or more purchases, at least one
within the past twelve months, and living six to ten miles from
- Male sporting goods buyers with just one order, 38+ months ago,
for under $15, and living eleven to fifteen miles from a store.
Clearly, these groups are no more difficult to understand than RFM
Cells. They will be just as easy to implement. But,
they will be much more stable than RFM Cells, and display superior
lift, because they are the product of a formal statistical process.
For Those Who Are Ready to Fully Embrace Statistics-Based
Predictive Models: Case Study #1
Although guarantees cannot
be made, predictive models often have the following impact when
they replace RFM Cells:
- A significant reduction in unprofitable contacts to single-buyers.
- A modest reduction in unprofitable contacts to multi-buyers, particularly
during weaker promotional seasons.
This is seen in Table's 10-A and 10-B, which illustrate the validation
results of single-buyer and multi-buyer predictive models that superseded
RFM Cells for a well-known direct marketer. Backend analysis
was performed on five external mailings that dropped subsequent
to the completion of the models. The analysis indicated that
— allowing for up to one anomaly — the following shaded
demi-deciles were below the direct marketer's $1.20 per piece mailed
Single-Buyer Model Validation
Multi-Buyer Model Validation
By replacing RFM Cells with statistics-based predictive models,
the direct marketer had the ability to generate substantial savings
by eliminating unprofitable contacts — especially to single-buyers,
and investing more heavily in lucrative customer relationship management
For Those Who Are Ready to Fully Embrace Statistics-Based
Predictive Models: Case Study #2
A profitable niche direct
marketer had successfully used RFM Cells for many years. A
statistics-based predictive model was constructed that was both
powerful and stable. Table 11 illustrates the model's performance.
With a breakeven of about $1.25 per piece mailed, and a re-mail
decline of about 50%, the customers on the validation file were
scored by the model, rank-ordered from highest to lowest predicted
performance, and grouped into the following deciles:
Customer Predictive Model
Customers in Decile 1 generated $8.14 per piece mailed, while Decile
10 brought in just $0.44. Combined, all of the deciles averaged
Deciles 8 through 10 are below the $1.25 breakeven, and were eliminated
from future promotions. The money that was freed up by eliminating
this unprofitable circulation was allocated to profitable re-mails,
and to initiatives such as the development of an Internet sales
Deciles 1 through 3 could be profitably re-mailed. Even with
a 50% performance drop-off, dollars per piece mailed remained over
$1.25. In contrast, RFM Cells were only able to identify about
15% of the database for re-mail treatment.
Decile 1 performed so well that it could be re-mailed twice.
That is because the resulting performance, even after a 75% drop-off,
remained comfortably above the $1.25 breakeven. In contrast,
the RFM Cells were unable to identify any customers who could be
For those of us who live in the Southeast, machetes
and the continual application of powerful herbicides are the only
effective antidotes to Kudzu. Direct marketers, however, can
eradicate antiquated RFM Cells for once and for all with statistics-based
predictive models that are more powerful, more stable, and infinitely
easier to implement.
Jim Wheaton is a Principal at Wheaton Group, and can be reached
at 919-969-8859 or firstname.lastname@example.org. The firm
specializes in direct marketing consulting and data mining, data
quality assessment and assurance, and the delivery of cost-effective
data warehouses and marts. Jim is also a Co-Founder of Data
 Regardless of the industry — catalog, retail, financial
services, telecommunications, fundraising, and the like —
the concepts to be discussed are the same.
 Some readers, mindful of the adage that 20% of the customers
generally account for 80% of the sales, will be skeptical of this
modest lift assumption. Keep in mind that, retrospectively,
we can identify with absolute certainty the best-performing 20%
of a given customer base, and then measure its performance.
What we are attempting to do here is different, and that is to predict
the best customers. Unfortunately, this can never be done
with anything near absolute accuracy. In other words, the
composition of our magical 20% is in constant flux, which degrades
significantly the often-quoted 80% as we look to the future.
 For reasons that will not be discussed here, it often is appropriate
to build multiple models.
 50% of 50%.