Click for a Printer-friendly Version
- Adobe PDF
Merge/Purge and DM Success
By Jim Wheaton and Cynthia Baughan Wheaton
Original Version of an article that appeared in the
March 4, 2002 issue of "DM News"
Have you ever wondered why so many rollouts are disappointing?
Three prior DM News articles established that a significant part
of the answer lies with statistical sampling theory ("How
Big Should My Test Be?," October 1, 2001, "Identifying
Millions in Lost Revenue," January 7, 2002, and "Why
So Many Rollouts Disappoint," February 4, 2002). In addition,
there are many non-statistical reasons, including changes in creative,
improper test design, list brokerage and management mix-ups, suboptimal
postal service performance, shifts in the economy, competitive developments,
and inconsistencies and inaccuracies in the merge/purge process.
This month, we will focus on the merge/purge process. More
precisely, we will revisit the topic by expanding on an article
last year that outlined how merge/purge can affect reported list
performance ("Merge/Purge Can Alter List Strategy,"
September 3, 2001).
Ignorance and Inattention
Since the late-1980's, merge/purge
has been a perceived commodity not worthy of attention by our industry's
movers and shakers. In such an environment, companies tend
to compete on price. And, all too often, direct marketers
get exactly what they are paying for in terms of quality.
In reality, merge/purge — at least for direct marketers who
rent large quantities of rental lists — is a series of intricate
and interconnected data processing steps. As such, every client
company should designate at least one knowledgeable person as having
overall responsibility for the process. This individual should
determine overall strategies as well as the ongoing details of execution.
Most clients, however, delegate these decisions to a low level employee
or to the service bureau itself. The disadvantages of the
in-house option are self-evident. However, delegating to the
service bureau can be equally problematic.
Often, service bureau account people operate as reactive order takers
rather proactive strategic partners. Generally, these individuals
have never been direct marketers, and view their assignments from
a very narrow processing perspective. Also, the demands of
the service bureau business tend to result in significant turnover,
especially among those in the lower ranks. And, these are
the same individuals who must ensure accuracy and consistency across
Steep Tax of Ignorance and Inattention
It is remarkable
how many times our firm, when beginning work with a new client,
discovers that formal, detailed merge/purge instructions do not
exist. Often, the sole guidance the service bureau is given
is to, "Run it just like last time." Frequently, when a careful
analysis is conducted, one or more major problems are uncovered.
The most egregious example occurred with a direct marketing company
whose merge/purge costs were approaching the outrageous level of
$70 per thousand names mailed. When the situation was analyzed,
the culprit turned out to be a suppression list that had grown wildly
out of control.
Several years earlier, the company had notified its service bureau
to suppress rental names that had been contacted during the previous
drop. The results were so positive that the company decided
to include such a suppression strategy in all subsequent merges.
However, the company did not supply its service bureau with explicit
instructions. Instead, the written guidance was to, "Run
it just like last time."
What the direct marketer meant was to suppress only the rental names
from the previous drop. What the service bureau did was create
a running suppression list. As impossible as it might seem,
this went on for many merge/purges until the suppression list had
grown to encompass many tens of millions of records. Needless
to say, a small fortune had been squandered in needless data processing,
and the direct marketer's available prospecting universe had
been severely and pointlessly constricted.
The parameters that drive a merge/purge,
and the circumstances that surround it, can affect the reported
results of a mailing — and, therefore, the probability of
rollout success. Because of this, the parameters should be
thoughtfully considered when setting up a job, and the underlying
circumstances factored in when conducting follow-up analysis.
As just one example, consider that the overall net rate out of the
record matching portion of a merge/purge usually drops as the number
of rental lists increases. This decreasing net rate, in turn,
is driven by an increasing percentage of prospects who appear on
multiple rental lists ("multis"). And, as a general
rule, multis have a response rate that is between 50% and 100% higher
than prospects who appear on just a single list ("uniques").
A hypothetical example will illustrate how all of this affects the
reported results of a mailing, and therefore can contribute to a
disappointing rollout. In this scenario, Prospect List A's
full universe of 100,000 is rented and then promoted without any
other outside lists. The resulting response rate is 1.01%,
which we will refer to as Prospect List A's "true"
response rate. Continuing with our example:
Subsequently, Prospect List A's full universe of 100,000 is
included in a merge/purge that results in a net rate out of record
matching of 80% for List A. For the sake of simplicity, we
will assume that no multis appear on more than two lists, which
means that 40% of the original 100,000 records are multis.
(In other words, with 40,000 multis, 20,000 will be attributed to
List A, and 20,000 will be attributed to other lists. The
result will be a net rate of 80,000 records for List A, or 80% of
the original 100,000.) We will also assume that multis do
twice as well as uniques.
The chart illustrates how, as the total input volume to the record
matching portion of the merge/purge increases, there is a direct
impact on the percentage of net names and the response rate for
List A. Specifically, as List A's response rate declines
to 0.90%, compared with its "true" rate of 1.01%, as
the net rate drops to 80%:
As a perceived commodity, merge/purge generally
is not considered worthy of concentrated focus. Direct marketers
pay for this with reduced circulation effectiveness. However,
a knowledgeable client-side contact, coupled with a trusted service
bureau partner, can transform the merge/purge into a powerful business
Jim Wheaton and Cynthia Baughan Wheaton are Principals at Wheaton
Group, and can be reached at 919-969-8859 or email@example.com.
The firm specializes in direct marketing consulting and data mining,
data quality assessment and assurance, and the delivery of cost-effective
data warehouses and marts. Jim is also a Co-Founder of Data