The magazine for the knowledge enabled enterprise

 A Bizmedia Publication

Knowledge Management Magazine

Home
Current Issue
Subscribe
Sample Copy
Feedback
Search
Contact Us

subscribe NOW!
Knowledge Management & Intranet Solutions - Conference & Exhibition
On-line Learning 2001 Europe


Tools of the trade

Campbell McCracken finds out whether a new OLAP really is emerging.

Most people think of OLAP (On-Line Analytical Processing) as being an old technology that hasn’t kept up with the times. There is some truth in this, but it depends on how you define OLAP. To some, OLAP is inseparable from pre-calculated cubes of multidimensional aggregated data. To others it is more than that, and pre-calculation is only one of many optimisation techniques.

When OLAP was first coined it meant multi-dimensional databases. "But along the way the term spread out and got abused," said Matthew Goldsbrough, Informatica’s European marketing director. "People who were providing more of a reporting tool or a relational tool jumped on the bandwagon and OLAP then became the term that was applied to any multi-dimensional analysis."

Traditional OLAP
Traditional OLAP tools achieve their on-line fast response through a number of techniques, including working on a fixed set of dimensions, aggregating the fine detail of the transaction data to reduce the scale of data, and pre-calculating results.

The fact that the finest granularity in an OLAP database is the aggregated transactional data means that you no longer have access to the original transactions. So if you need information that the database hasn’t been constructed to provide, you have to re-populate it.

"One of the things that people got fed up with with OLAP is that if the IT people who built the OLAP database hadn’t pre-sought out everything that they would want to query, then they would have to go back to the beginning and rebuild it," said Debbie Atherton, technical director at MarkIT Information Services Limited (MarkIT). "They were only good as long as you never wanted to ask anything that hadn’t previously been thought of."

Alterian and the New Breed
One vendor that offers a new breed of on-line processing tools is Alterian. Its products give you the traditional analytical processing capabilities plus access to the lowest level of transactional data but without losing out on speed. These were both design aims at the outset.

"We had to have equal speed on unknown information or unknown questions," said Alterian’s Anthony Power. "And that is speed on the unaggregated, lowest level granular data that you have. We achieve speed through innovative techniques of dealing with the raw data. We do not aggregate data like an old OLAP tool does."

Alterian took the further step of making its tools into an application development environment rather than just a database. "We offered [partners] an application environment so they could build an application around a specific process," said Power, "and a high speed engine that allowed them to answer the kinds of questions that they needed to answer that a reporting / OLAP tool was not, in their minds, appropriate for."

Alterian works through partners in three segments. The first is marketing services companies who have huge client databases and who have the objective of adding value to the data that they either sell or manage. The second segment is the systems integrators who build custom projects for clients, such as EDS and Dimension Data. Finally Alterian sells to technology companies who want to embed its engines in order to broaden the functionality & capability of their suite of tools.

Rather than producing a complete set of solutions Alterian supplies the environment, the tools, and template examples. "What we’re trying to do is jump-start some of those solutions because it is our belief that our partners will know more about those specific application areas than we will," said Power. "We want to be able to focus more on the core technology side, but do recognise that we have to educate and illustrate the art of the possible to our partners."

The customer
With the advent of the Electronic Point of Sale (EPOS) systems and loyalty programs, retail companies have more information at their disposal. And not only is there a vast difference in the quantity of information that’s now available, it’s a different sort of information. For the first time it’s not information about the business.

"All of a sudden you had tremendously detailed information and the biggest change was the injection of customer records," said Alterian’s US vice-president of operations, Anthony Power. "For the first time people started to have data about ‘the customer’."

Armed with this information, companies started asking different questions of the databases. Instead of the analysis only being carried out by managers wanting to find out the state of the business, marketers now wanted to find out why people were buying. "The fact that I know everything about, say, how many shoes I sold, where I sold them, who bought them and what it cost, does not tell me what are the principal drivers in shoe sales" said author, lecturer and consultant Erik Thomsen.

New questions

Case Study
UK domestic energy supplier Amerada wanted to build a marketing database to help drive efforts across all its marketing channels. It wanted to be able to increase customer acquisitions and retention and to be able to cross-sell and up-sell. More importantly, it wanted to do these profitably and to be able to assess long term profitability.

GB Information Management provided a solution for Amerada based on Alterian’s database technology. "The Alterian product comes with a pre-packed set of reports, but we also developed tailored reports using VBA," said GB Information Marketing’s head of analysis and consultancy, Dene Jones.

Alterian was chosen in preference to a traditional OLAP solution. "We did have an OLAP solution as part of our suite," said Jones, "but there was a lot of preconditioning we needed to do, things like the creation of cubes. This meant that the analysis was limited in terms of what the client could do."

"The client was constrained to only getting answers to the questions they were going to ask on a regular basis. There was no ability to do very much ad-hoc train of thought querying because in many cases the cubes weren’t set up to enable that easily. With Alterian they can query at any level they choose, at any time they want to."

What the marketers now want to be able to do to is find out more about customer behaviour. The famous case of such behaviour was the discovery by convenience stores that there was a correlation between sales of nappies and beer. The driver behind this was that husbands were being sent out to buy nappies, and while they were in the store they decided to buy themselves some beer. This led to the stores arranging their displays to put beer and nappies closer together to take advantage of this correlation, or indeed encourage it.

"The question became ‘Find me a behaviour that I’m interested in’," says Alterian’s Power. "I want to replicate a good behaviour amongst more people, so find me people like the people who exhibited that good behaviour." This new type of question requires finer detail in the database and a different analysis. For one thing, it needs access to the transactional information that has been aggregated out of the OLAP database.

Another area where the real-time aspect of the analysis is being sought is in ecommerce-based applications. "What they want to be able to do is [perform the analysis] real-time and have it sent back to the customers as it’s going along, so that it’s a closed loop," says ETI’s chief technology officer David Marshall. "Good examples of that are people like Amazon who have their own proprietary way to be able to have a loop to the customer of what are the hottest products, what other people are saying etc."

However these are not tasks that traditional OLAP is good at. "It’s not the people who are necessarily moving away from OLAP, which would imply that it doesn’t have a purpose and doesn’t have a use, because I don’t believe that is true," said Alterian’s Anthony Power. "What has happened is that there is a new set of questions that are being asked for which OLAP may not be the right technology."

OLAP is dead. Long live OLAP.
New tools were developed to enable this sort of analysis to be performed. And if you subscribe to the wider definition, these are still OLAP tools. "[Marketing-led questions] are highly dimensional questions, so they may not be appropriate for a simplistic, pre-calculate-oriented OLAP tool," says Erik Thomsen, "But they are absolutely admittable to appropriate multi-dimensional modelling analysis."

Matthew Goldsbrough, Informatica’s European marketing director

So now there is a new OLAP, although not many people refer to it by that name because of the overtones that it carries. But as Erik Thomsen points out, "OLAP has nothing to do with pre-calculation. Pre-calculation is one very simplistic optimisation technique."

Other companies use different techniques. Database migration company ETI helps partner companies by reducing the amounts of data that have to be analysed. "We’re sending them smaller subsets," says ETI’s Marshall. "Our technology makes sure that we’re only selectively retrieving information so that you’re really only getting small bits of data if you need it, as opposed to having to dump large files and to process them through."

Other companies, such as Informatica, provide a wide range of offerings. "We have a combination of packaged applications and underlying technology that enables people to pull unknown information from many sources, bring that together and then analyse it for a large number of users throughout the enterprise" said Matthew Goldsbrough.

Some vendors are using multi-dimensional functionality in embedded systems and embedded software. "I’ve seen applications where the word OLAP doesn’t appear in the solution but if you look at it, it’s built on top of a known OLAP tool, or multi-dimensional tool," says Thomsen.

What is the future
The need for fast analysis tools in the future looks set to continue. "In all the studies that you see," said Goldsbrough, "the one that sticks in my mind is something that the University College of Berkeley did where it calculated that the data that is going to be managed over the next two years is going to be more voluminous than all the data that had been under management in the past 40 years.

"So there’s a huge exponential explosion in the supply of raw data coupled with a huge growth in the number of people that are trying to use it. You need something in the middle that can make the transition from vast amounts of data into something that’s usable by that growing band of humans."

One trend we are likely to see is the increased use of textual information being used in analysis. "There’s recognition now that incorporating non-numeric data can help build predictor models," says Erik Thomsen. "So predictor models for something as mundane as cocoa futures can be improved if you’re scanning press releases and news bulletins, and incorporating that into your predictor models.

"If you can figure out just a little better than the next guy whether or not it makes sense to spend $300m on that patch of sea so that you can drill oil there, you’re gonna do pretty well. Small increments in your ability to predict have huge returns."

There’s still life in the OLAP dog
However the new analysis tools look unlikely to replace the traditional OLAP tools for business analysis. "It’s more of a complementary strategy in most companies," said Alterian’s Anthony Power. Erik Thomsen agreed. "Very few tools, if any, are really fully adequate for the entire range."

And despite their running out of steam in some areas, they still do their job well in their segment. "OLAPs are still good," said Debbie Atherton, "They’ve just been surpassed."

 

       

 

© 2001 Bizmedia Ltd under licence from Learned Information Europe Ltd

previous pagetop of page