Click for a Printer-friendly Version
- Adobe PDF
Workstation Technology for Marketing Analysis
By Boris Gendelev
Principal, Wheaton Group
Original version of an article that appeared in the January 1990
[Note: Despite dramatic increases in raw computing power and
a proliferation of end-user software tools since the publication
of this article, virtually all of the content remains highly relevant.
Interestingly, this article recognized the major reasons for, and
concepts of, data warehousing before the idea was popularized and
gained acceptance. It also anticipated the widespread use
of microprocessor workstations for OLAP.]
Analytical workstations: are they for you? If you are
having trouble getting a simple analytical report, the answer may
Nothing is more valuable to a marketing manager than timely and
accurate information. Yet many managers are frustrated, unable to
get even a simple analytical report. We believe this is because
companies are often slow to accept the solutions new technologies
A spectrum of analytical concepts, such as segmentation, lifetime
value, product affinity and channel interaction, have been developed
for the purpose of extracting the most relevant information from
historical data. Yet, too often, attempts to incorporate these
techniques stall, creating friction between marketing and systems
professionals. The technological foundation of successful
marketing analysis is still a mystery to many managers. Only
when its unique aspects are brought to the surface does the need
for an entirely new approach become more apparent.
Marketing analysis can be described as a search for actionable information
in a sea of historical data, guided by general business concepts
and conjectures. It is by nature iterative, driven by the
data themselves. Data analysis requires focusing intently;
one insight may lead to another, and there is always an urgent need
to know if you are on the right track. Yet, as with everything
else, there are budgets and deadlines.
To use powerful analytical marketing techniques with consistent
success, one must have a foundation of:
- Usable data.
- Flexible data manipulation tools.
- Responsive processing environment.
- Predictable turn around time.
- Low cost computer resources.
- Ability to implement findings.
Contrasted to the traditional approach, this article introduces
the evolving technology of microprocessor based analytical workstations.
In our view, this technology meets the information needs of marketing
management by addressing the fundamentals.
Traditional data processing historically evolved around mainframes
and, more recently, minicomputers." MIS departments and
service bureaus have standardized around a relatively small set
of technologies and problem-solving methods. Unfortunately
these environments are often unsuitable for marketing analysis tasks:
- The emphasis is on smooth and reliable support of a company's daily
operations. Anything else that takes up resources is regarded
as a nuisance and given low priority.
- Data files are structured to optimize daily transaction processing.
A minimum amount of data is kept on-line.
- Few procedures or incentives exist to ensure correctness of data
that are not part of the operational cycle, but which might be vital
to marketing (e.g., source and offer codes).
- Developing applications requires mounds of detailed specification
and a lengthy formal review process.
- Flexibility, when it is built into the systems, takes the form of
extra "buckets" — an approach that only delays obsolescence.
- Personnel, trained to be efficient in a highly structured environment,
have difficulty adjusting to midstream changes.
- There is resistance to new, untried technologies.
- Users are charged per unit of processing. This mechanism works
well only if processing needs are easy to estimate.
- Software packages, particularly those targeted at high-end marketing
users (e.g., MORE/2, and Metaphor) are expensive.
In contrast, the process of marketing data analysis:
- Is impossible to specify completely in advance.
- Often is time critical.
- Demands access to a lot of historical data.
- Requires availability of significant computer resources.
- Can benefit from new technologies such as expert systems and neural
Chart A summarizes the characteristics of traditional data processing
and how they impede successful data analysis:
The conflict between marketing needs and the realities of data processing
impacts all stages of data analysis — from data preparation
and modeling to implementation.
The need for data preparation itself arises for the most part because:
- The structure of data must be changed to fit marketing perspective.
For instance, it is common to find shipping and handling charges,
discounts, premiums and taxes as records in a line items file.
From an analyst's viewpoint, this information should be on
- Important fields, such as source and offer codes, are missing or
incorrect, sometimes in more than 50 percent of cases.
- Duplication is done only on mailing records, not on the main database.
While most of the challenges can be overcome, it often takes a substantial
iterative effort, affording little time to develop formal specifications.
As early the data preparation stage, the hands-on involvement of
a marketing data analyst is needed, but is usually obstructed by
the absence of tools that would overcome lack of programming skills.
The statistical modeling itself requires programming, although,
with luck, using a high-level language. The desired features of
such a language would include:
- Ability to pull together and process as cases all pieces of information
pertaining to a customer, such as demographics, promotions, orders,
line items and payments.
- Easy ways of creating new customer categories from that rich variety
- Aggregation of statistics across cases by existing and newly created
Unfortunately no popular database query or statistical analysis
package (e.g., SQL, FOCUS, SPSS, and SAS) adequately supports all
of these functions and most suffer from poor performance.
Even if the analyst can program in FORTRAN, COBOL or any other lower-level
language, debugging would be very time-consuming, and would have
to be done every time something even slightly different is contemplated.
As a result of not having powerful enough data manipulation tools,
the data preparation stage is often extended to produce a file easily
manipulated by an existing package such as SPSS or SAS. That,
however, means committing to premature data aggregation (usually
by creating customer summaries and indicators), thereby losing analysis
flexibility or having to redo data preparation several times, incurring
additional processing charges and making turnaround time unpredictable.
As a rule, the more problems encountered in data preparation, the
more difficult it is to implement any kind of segmentation or forecasting
in the data processing environment. Tactical analyses, such
as R/F/M-based circulation planning or SKU demand forecasting, have
a direct impact on a company's data processing because explanatory
variables have to be reliably recreated on the main file and tracking
mechanisms need to be set in place.
Long-term-oriented analyses such as lifetime value or product affinity
usually make their impact on the company more gradually through
changes in marketing strategies, making implementation a bit easier.
To summarize, a traditional data processing environment is usually
ill suited to meet the needs of marketing data analysis. To
expect of it effective and efficient support of marketing management
is like expecting a marathon winner to compete and win in sprints.
Microcomputer-Based Analytical Database Approach
Powerful microcomputers are an excellent platform for marketing
data analysis, even on a large scale. In fact, a high-end
micro has more power potential storage capacity and software than
some minicomputers. Moreover, PCs are getting more powerful
much faster than mainframes or minis. And all of its resources
are usually dedicated to and controlled by a single user.
Our experience shows that it is possible to run an R/F/M analysis
on a database with half a million customers, one million orders
and five million line items in a mere thirty minutes.
The cost per unit of processing on a PC is about one-thirtieth of
that on a mainframe and one-tenth of the cost on a typical minicomputer.
The cost per unit of disk storage is about half. While hardware
costs are falling overall, the relative cost gap is only getting
The notion of performing data analysis in a dedicated computing
environment goes hand in hand with the notion of maintaining a comprehensive
analytical database with its own set of support programs and manipulation
tools. The availability of a wide range of easy-to-use microcomputer
software, from template-based generators to expert system shells
and object-oriented languages, makes this feasible. The advantages
- The process of its construction will force a systematic review of
available data and produce long-term solutions to make it more usable.
- Once constructed it will require only relatively infrequent updates,
thus minimizing the dependency on the main system.
- Performance can be optimized to a set of very specific needs.
- Initial investment in equipment is low and incremental processing
costs are close to zero.
- With the help of specially tailored tools, new analytical ideas
can be tried immediately, without involving data processing.
- Many interrelated analyses can be performed on the same consistent
- Customer mailing selection and keycoding can be easily preformed
from such databases, integrating these tasks with analysis.
Chart B summarizes benefits of the microcomputer workstation approach
in relation to the needs of successful analysis:
This new approach is counter to recent trends toward closer systems
integration. However, we have to remember that a marketing
analysis system, while sharing terminology with a marketing tracking
system, is different because it serves the planning rather than
control function in a company. It would benefit little from
being embedded into one super system along with fulfillment, inventory
control and accounting. In most cases, a selection and transfer
of customer records would be a satisfactory level of interface to
a mailing system. While other systems can be impacted by results
of data analysis, periodic feedback is sufficient.
The success of the new approach stems from matching appropriate
technologies to the unique nature of marketing analysis. To
be manageable, analytical databases must be organized with minimal
redundancy and maximum useful transaction detail. Experience
shows that the marketing analysis database will be just one-twentieth
to one-fifth the size of the company's main database.
A well-conceived database construction sets the stage for fast economical
analysis by putting the data into a ready to use form. The
task of restructuring, reforming, rematching and recoding the data
should now be viewed as building a permanent interface between the
data processing function and the analytical database. It is
best done by someone outside the data processing department, who
appreciates marketing analysis needs and has good computer skills.
Once the analytical database is constructed, flexible manipulation
of the data can be accomplished by constructing specialized high-level
tools that have the following functions:
- Aggregation of counts and amounts by one or more categories in the
form of frequencies, cross-tabs and means. For example, quarterly
demand and orders by source and data of first purchase.
- Ability to apply statistical procedures such as regression.
- Flexibility for a general programming language for specifying transformations
of variables, such as recoding or categorization.
- Support for multiple record types (e.g., customer, order, item)
within each customer case. This means an ability to easily
construct new categories by formulating questions such as:
- "How much did a customer spend last year?"
- "How many times did a customer buy in given product category?"
- "What was the main product category of a customer's
- "What product families did a customer purchase from last
- "What combinations of product families does a customer buy
- "Was a recently bought product purchased before?"
- Access to reference tables. This might be needed for assignment
of product families based on SKU, for instance.
- Easy interface to other software such as spreadsheets, report generators
and presentation graphics.
- Ability to create mailing selections.
These capabilities can be either developed or acquired at a cost
often substantially lower than creating a comparable functionality
in a traditional data processing environment.
From our experience in managing analytical databases we can report
that the appetite for information grows with the ability to access
it easily and inexpensively. One apparel cataloger at first
wanted to know lifetime customer activity patterns by source, then
also by size and type of the first transaction. Recently, queries
evolved to include: "Do customers buy tops and bottoms
together and what color combinations do they buy?" "Is
there a consistent preference for a color?" "What
is the long-term effect of backorders and unfulfilled orders?"
What began as a wish list is now an integral part of their marketing
In planning, strategic or tactical, information is everything.
Marketing analysis is a tool of planning. In a climate of increased
competition and rising costs, one cannot afford to over1ook new
and promising information technology. If your corporate goals
include smarter customer acquisition and development, improved efficiency
of promotions, better pricing and coordination of marketing channels,
do not overlook analytical workstations.
Boris Gendelev is a Principal at Wheaton Group, and can be reached
at 847-205-0916 or firstname.lastname@example.org.
The firm specializes in direct marketing consulting and data mining,
data quality assessment and assurance, and the delivery of cost-effective
data warehouses and marts.