Tools of the trade |
| |
|
|
Campbell McCracken
finds out whether a new OLAP really is emerging.
Most
people think of OLAP (On-Line Analytical Processing) as
being an old technology that hasn’t kept up with the
times. There is some truth in this, but it depends on
how you define OLAP. To some, OLAP is inseparable from
pre-calculated cubes of multidimensional aggregated
data. To others it is more than that, and
pre-calculation is only one of many optimisation
techniques.
When
OLAP was first coined it meant multi-dimensional
databases. "But along the way the term spread out and
got abused," said Matthew Goldsbrough, Informatica’s
European marketing director. "People who were providing
more of a reporting tool or a relational tool jumped on
the bandwagon and OLAP then became the term that was
applied to any multi-dimensional analysis."
Traditional OLAP Traditional OLAP tools achieve their on-line fast
response through a number of techniques, including
working on a fixed set of dimensions, aggregating the
fine detail of the transaction data to reduce the scale
of data, and pre-calculating results.
The
fact that the finest granularity in an OLAP database is
the aggregated transactional data means that you no
longer have access to the original transactions. So if
you need information that the database hasn’t been
constructed to provide, you have to re-populate it.
"One
of the things that people got fed up with with OLAP is
that if the IT people who built the OLAP database hadn’t
pre-sought out everything that they would want to query,
then they would have to go back to the beginning and
rebuild it," said Debbie Atherton, technical director at
MarkIT Information Services Limited (MarkIT). "They were
only good as long as you never wanted to ask anything
that hadn’t previously been thought of."
Alterian and the New
Breed One vendor
that offers a new breed of on-line processing
tools is Alterian. Its products give you the
traditional analytical processing capabilities
plus access to the lowest level of transactional
data but without losing out on speed. These were
both design aims at the outset.
"We had to have equal speed on unknown
information or unknown questions," said Alterian’s
Anthony Power. "And that is speed on the
unaggregated, lowest level granular data that you
have. We achieve speed through innovative
techniques of dealing with the raw data. We do not
aggregate data like an old OLAP tool
does."
Alterian took the further step of making
its tools into an application development
environment rather than just a database. "We
offered [partners] an application environment so
they could build an application around a specific
process," said Power, "and a high speed engine
that allowed them to answer the kinds of questions
that they needed to answer that a reporting / OLAP
tool was not, in their minds, appropriate
for."
Alterian works through partners in three
segments. The first is marketing services
companies who have huge client databases and who
have the objective of adding value to the data
that they either sell or manage. The second
segment is the systems integrators who build
custom projects for clients, such as EDS and
Dimension Data. Finally Alterian sells to
technology companies who want to embed its engines
in order to broaden the functionality &
capability of their suite of
tools.
Rather than producing a complete set of
solutions Alterian supplies the environment, the
tools, and template examples. "What we’re trying
to do is jump-start some of those solutions
because it is our belief that our partners will
know more about those specific application areas
than we will," said Power. "We want to be able to
focus more on the core technology side, but do
recognise that we have to educate and illustrate
the art of the possible to our
partners." |
The
customer With
the advent of the Electronic Point of Sale (EPOS)
systems and loyalty programs, retail companies have more
information at their disposal. And not only is there a
vast difference in the quantity of information that’s
now available, it’s a different sort of information. For
the first time it’s not information about the business.
"All
of a sudden you had tremendously detailed information
and the biggest change was the injection of customer
records," said Alterian’s US vice-president of
operations, Anthony Power. "For the first time people
started to have data about ‘the customer’."
Armed with this information, companies started
asking different questions of the databases. Instead of
the analysis only being carried out by managers wanting
to find out the state of the business, marketers now
wanted to find out why people were buying. "The fact
that I know everything about, say, how many shoes I
sold, where I sold them, who bought them and what it
cost, does not tell me what are the principal drivers in
shoe sales" said author, lecturer and consultant Erik
Thomsen.
New
questions
Case Study UK domestic energy supplier Amerada
wanted to build a marketing database to help drive
efforts across all its marketing channels. It
wanted to be able to increase customer
acquisitions and retention and to be able to
cross-sell and up-sell. More importantly, it
wanted to do these profitably and to be able to
assess long term profitability.
GB Information Management provided a
solution for Amerada based on Alterian’s database
technology. "The Alterian product comes with a
pre-packed set of reports, but we also developed
tailored reports using VBA," said GB Information
Marketing’s head of analysis and consultancy, Dene
Jones.
Alterian was chosen in preference to a
traditional OLAP solution. "We did have an OLAP
solution as part of our suite," said Jones, "but
there was a lot of preconditioning we needed to
do, things like the creation of cubes. This meant
that the analysis was limited in terms of what the
client could do."
"The client was constrained to only getting
answers to the questions they were going to ask on
a regular basis. There was no ability to do very
much ad-hoc train of thought querying because in
many cases the cubes weren’t set up to enable that
easily. With Alterian they can query at any level
they choose, at any time they want
to." |
What
the marketers now want to be able to do to is find out
more about customer behaviour. The famous case of such
behaviour was the discovery by convenience stores that
there was a correlation between sales of nappies and
beer. The driver behind this was that husbands were
being sent out to buy nappies, and while they were in
the store they decided to buy themselves some beer. This
led to the stores arranging their displays to put beer
and nappies closer together to take advantage of this
correlation, or indeed encourage it.
"The
question became ‘Find me a behaviour that I’m interested
in’," says Alterian’s Power. "I want to replicate a good
behaviour amongst more people, so find me people like
the people who exhibited that good behaviour." This new
type of question requires finer detail in the database
and a different analysis. For one thing, it needs access
to the transactional information that has been
aggregated out of the OLAP database.
Another area where the real-time aspect of the
analysis is being sought is in ecommerce-based
applications. "What they want to be able to do is
[perform the analysis] real-time and have it sent back
to the customers as it’s going along, so that it’s a
closed loop," says ETI’s chief technology officer David
Marshall. "Good examples of that are people like Amazon
who have their own proprietary way to be able to have a
loop to the customer of what are the hottest products,
what other people are saying etc."
However these are not tasks that traditional OLAP
is good at. "It’s not the people who are necessarily
moving away from OLAP, which would imply that it doesn’t
have a purpose and doesn’t have a use, because I don’t
believe that is true," said Alterian’s Anthony Power.
"What has happened is that there is a new set of
questions that are being asked for which OLAP may not be
the right technology."
OLAP
is dead. Long live OLAP. New
tools were developed to enable this sort of analysis to
be performed. And if you subscribe to the wider
definition, these are still OLAP tools. "[Marketing-led
questions] are highly dimensional questions, so they may
not be appropriate for a simplistic,
pre-calculate-oriented OLAP tool," says Erik Thomsen,
"But they are absolutely admittable to appropriate
multi-dimensional modelling analysis."
|
Matthew
Goldsbrough, Informatica’s European marketing
director |
So
now there is a new OLAP, although not many people refer
to it by that name because of the overtones that it carries. But as Erik Thomsen
points out, "OLAP has nothing to do with
pre-calculation. Pre-calculation is one very simplistic
optimisation technique."
Other companies use different techniques.
Database migration company ETI helps partner companies
by reducing the amounts of data that have to be
analysed. "We’re sending them smaller subsets," says
ETI’s Marshall. "Our technology makes sure that we’re
only selectively retrieving information so that you’re
really only getting small bits of data if you need it,
as opposed to having to dump large files and to process
them through."
Other companies, such as Informatica, provide a
wide range of offerings. "We have a combination of
packaged applications and underlying technology that
enables people to pull unknown information from many
sources, bring that together and then analyse it for a
large number of users throughout the enterprise" said
Matthew Goldsbrough.
Some
vendors are using multi-dimensional functionality in
embedded systems and embedded software. "I’ve seen
applications where the word OLAP doesn’t appear in the
solution but if you look at it, it’s built on top of a
known OLAP tool, or multi-dimensional tool," says
Thomsen.
What
is the future The
need for fast analysis tools in the future looks set to
continue. "In all the studies that you see," said
Goldsbrough, "the one that sticks in my mind is
something that the University College of Berkeley did
where it calculated that the data that is going to be
managed over the next two years is going to be more
voluminous than all the data that had been under
management in the past 40 years.
"So
there’s a huge exponential explosion in the supply of
raw data coupled with a huge growth in the number of
people that are trying to use it. You need something in
the middle that can make the transition from vast
amounts of data into something that’s usable by that
growing band of humans."
One
trend we are likely to see is the increased use of
textual information being used in analysis. "There’s
recognition now that incorporating non-numeric data can
help build predictor models," says Erik Thomsen. "So
predictor models for something as mundane as cocoa
futures can be improved if you’re scanning press
releases and news bulletins, and incorporating that into
your predictor models.
"If
you can figure out just a little better than the next
guy whether or not it makes sense to spend $300m on that
patch of sea so that you can drill oil there, you’re
gonna do pretty well. Small increments in your ability
to predict have huge returns."
There’s still life in the OLAP
dog However the new analysis tools look unlikely to
replace the traditional OLAP tools for business
analysis. "It’s more of a complementary strategy in most
companies," said Alterian’s Anthony Power. Erik Thomsen
agreed. "Very few tools, if any, are really fully
adequate for the entire range."
And
despite their running out of steam in some areas, they
still do their job well in their segment. "OLAPs are
still good," said Debbie Atherton, "They’ve just been
surpassed."
|