Attaining Real Time, On-demand Information Data: Contemporary Business Intelligence Tools

Pure-Play BI Vendors

Some pure play business intelligence (BI) vendors have long been providing BI platforms, which offer complete sets of tools for the creation, deployment, support, and maintenance of BI applications. Pure-play vendors attempt to sell these platforms to original equipment manufacturers (OEM) and independent software vendors (ISV), and even to IT organizations and end users that are information technology (IT) savvy enough to build their own applications on top of it. These BI platforms logically combine many database access capabilities like structured query language (SQL), online analytical processing (OLAP) data manipulation, modeling functions (what-if analysis), statistical analysis, and graphical presentations of results (charting) to create data-rich applications. The applications have customized user interfaces (UI), and are organized around specific business problems that target business analyses and models.

Part Three of the Business Intelligence Status Report series.

Most BI platform vendors also offer their own BI applications as the BI platform's validation. These enterprise BI application suites are the descendants of basic query-and-reporting tools, which they tend to displace or extend. The suites provide support for varying levels of users, with a variety of query, reporting and OLAP capabilities, all available with the idea of minimal training.

The BI market is currently crowded with a number of vendors with adept product suites. Some players in the market include MicroStrategy, Informatica, Information Builders, Oracle, IBM, SAP, Microsoft, Teradata, and Ascential, which was reunited with its short-term foster parent Informix, when Ascential was acquired by IBM, and See IBM Buys What's Left of Informix). Other vendors include Applix, arcplan, ProClarity, Siebel Systems, OutlookSoft, CombineNet, and SPSS. Obviously, these vendors have different origins. Some are traditional database vendors and enterprise application vendors; others were BI suite vendors, pure players in certain niches (such as enterprise reporting), while some have evolved to cover multiple bases.

Lately most of these vendors have updated their client/server tools with a Web-based UIs. Many BI software providers create standards-based portlets, sometimes using Web services, to expose BI functionality and information when more than viewing is required, and to be consistent with other portal add-ins. The advantage of portlets over simple hyperlinks (which have initially been leveraged) is a richer, more interactive experience and a more uniform integration approach. Otherwise, a portlet is a Web-based component that will process requests and generate dynamic content. The end user essentially sees a portlet as being a specialized content area within a Web page that occupies a small window in the portal page. The portlet provides users with the capability to customize content, with the appearance and position of a portlet.

Incidentally, current portlet standardization might help Web enablement and portal environment integration efforts, as most BI vendors are currently and cautiously developing products that will adhere to JSR 168, a standard that enables portlet interoperability. Since a JSR 168-compliant portal should be compatible with all Java-based portal solutions, it should alleviate the development burden on BI vendors. As a result, vendors can then focus on more advanced BI functionality rather than on porting their solutions to the individual, commercially available enterprise portal products. Thus, because of a strong Web similarity associated with BI application suites, some vendors describe their offerings as BI portals, whereby these portal offerings typically provide a subset of the counterpart client/server functionality via a Web browser. However, the vendors have been steadily increasing this functionality to come closer to that provided by rich, Microsoft Windows-like client desktop tools.

This is Part Three of a seven-part note.

Part One detailed history and current status.

Part Two looked at contemporary BI tools.

Part Four will describe the BI/CPM market landscape.

Part Six will discuss Geac and Point Solutions vendors.

Part Six will compare direct access to a data warehouse for the mid-market.

Part Seven will make recommendations.

Is Real Time, On-Demand BI Attainable?

There are those who always want more due to the many pressures that demand the need and investment in data management and integration. The trend of massive growth in data volumes continues with no end in sight, while enterprises have to manage and share large amounts of data across diverse regions and lines of businesses (LOB). The introduction of new data generating technologies, such as radio-frequency identification (RFID), will only accelerate this growth and the subsequent need for real time BI. Traditional BI systems use a large volume of static data that has been extracted, cleansed, and loaded into a data warehouse (DW) to produce reports and analyses. However, the need is not just reporting, since users need business monitoring, analysis, and an understanding of why things are happening.

The demand for instant, on-demand access to dispersed information has grown as the need to close the gap between the operational data and strategic objectives has become more pressing. As a result, a category of products called real-time BI applications have emerged. These can provide users, who need to know (virtually in real-time) about changes in data or the availability of relevant reports, alerts, and notifications regarding events and emerging trends in Web, e-mail, or instant messaging (IM) applications. In addition, business applications can be programmed to act on what these real-time BI systems discover. For example, a supply chain management (SCM) or enterprise resource planning (ERP) application might automatically place an order for more "widgets", for example, when real time inventory falls below a certain threshold, or when a customer relationship management (CRM) application automatically triggers a customer service representative and credit control clerk to check a customer who has placed an on-line order larger than $10,000.

The first approach to real time BI uses the DW model of traditional BI systems. In this case, products from innovative BI platform providers like Ascential or Informatica provide a service-oriented, near real time solution that populates and the DW much faster than the typical nightly extract/transfer/load (ETL) batch update does. The second, commonly called business activity monitoring (BAM) is adopted by pure play BAM and or hybrid BAM-middleware providers such as Savvion, Iteration Software, Vitria, webMethods, Quantive, Tibco (particularly after the acquisitions of Staffware and Praja) or Vineyard Software. It bypasses the DW entirely and uses Web services or other monitoring means to discover key business events. These software monitors or agents can be placed on a separate server in the network or on the transactional application databases themselves, and they can use event- and process-based approaches to proactively and intelligently measure and monitor operational processes.

For more on these diagnostic BI tools, see Business Activity Monitoring—Watching the Store for You. The most advanced of these applications not only optimize users' time and the information they receive, but also provide the context for them to take appropriate action.

EII Complements or Renders DW Obsolete?

Enterprise information integration (EII) is an emerging category of software that confronts the longstanding challenge of enterprise data integration over diverse data sources in scattered enterprise systems. Companies that have overcome the problem of scaling and managing data are now pondering how to unify their data sources and leverage them to solve near, real time business problems. To that end, EII aims at providing unified views of multiple, heterogeneous data through a distributed (federated) query. One way to think of EII is as a virtual database layer that allows user applications to access and query data as if it resided in a single database. In other words, the concept has taken an existing database capability to merge a query across different tables, but it is done on a virtual basis, shielding users from the underlying complexities of locating, querying, and joining data from varied data source systems.

EII is a fundamentally different approach to other data integration technologies such as enterprise application integration (EAI), which provides data or process-level integration, and enterprise portals, which merely integrate data at the presentation level. To refresh our memory, EAI is the unrestricted sharing of data and business processes throughout networked applications or data sources in an organization.

Early enterprise applications in areas such as, inventory control, human resources (HR) and payroll management, sales force automation (SFA), and database management system (DBMS), were designed to run independently, with no interaction between the systems. They were custom-built within the technology of the day for a specific need, and were often proprietary systems. As enterprises grew, they recognized the need for information and applications to be transferred across and shared between systems. As a result, enterprises often invested in EAI in order to streamline processes and keep all the elements of the enterprise interconnected.

There are four major categories of EAI:

1. Database linking, whereby databases share information and duplicate information as needed.
   
   
2. Application linking, whereby the enterprise shares business processes and data between two or more applications.
   
   
3. Data warehousing whereby data is extracted from a variety of data sources and channeled into a specific database or DW for analysis.
   
   
4. Common virtual system, which would be the peak of EAI, whereby all aspects of enterprise computing are tied together so that they appear as a unified application.

EII is also differentiated from the conventional ETL tools for data warehousing because it neither moves data nor creates new data stores of integrated data. Rather, it leaves data where it is, leveraging metadata repositories across multiple foundation enterprise systems and visibly pulls information into new applications. As a result, customers may be content to trade-in expensive and pesky DWs for a data extraction and presentation layer that sits on top of existing transactional systems, but only on the condition that they receive unimpaired performance. As a result, this will make virtual or abolish the intermediary step requiring diverse data sources to be aligned and their terms of use to be agreed upon.

Another way to look at the EII approach, somewhat borrows from material management approaches. EAI and ETL can be thought of as "push" technologies, and EII can be regarded as a "pull" mechanism that seeks and finds data, as needed and in near real-time, by creating an enterprise-wide abstraction semantic layer for standardized access to any corporate data source. The ability to provide appropriate BI without having to adapt a DW for specific decision support tasks is sometimes referred to by EII vendors as on-demand BI.

Companies can certainly benefit from accelerating BI analysis and reporting, to expedite end-of-quarter closing reports, to incorporate regulatory compliance efforts. EII may offer opportunities for more effective data management, provisioning, and auditing within "a single version of the truth". Also, near real time information can be especially useful for certain industries characterized by data diversity. For example, an airline might need to know specific passenger information as soon as the plane pulls away from the gate, or a retailer may need real time information to plan inventory distribution, allowing hourly metrics to guide selling and supply strategies. BI analysis and reporting can combine demographics data, and summary and line item purchasing histories from data stores and point-of-sale (POS) systems to aid customer service applications designed to deliver a 360-degree view of customers. Other industries that can benefit include the insurance and pharmaceutical industries. For example, an insurance field adjuster can reference customer claim forms and other account information residing in disparate databases. Last but not least, pharmaceutical and life science companies can assemble patient data, scientific information, clinical trial information, and feeds coming from various proprietary sources. Moreover, for some environments, it might make more sense to leave the data where it is, pulling them in to create a consolidated view when needed rather than lumping them into one massive database.

Other useful EII (real-time, on-demand BI) deployments could be within operational dashboards to track various performance metrics, or in financial risk analysis, where each single transaction might affect a serious change. The advent of Web services and Internet standards, such as extendable markup language (XML) and simple object access protocol (SOAP) will certainly help with the future integration of data and with real time BI. Enterprises considering a service-oriented architecture (SOA) could embed an EII server that would publish data integration as a Web service. For more pertinent information, see Understanding SOA, Web Services, BPM, BPEL, and More.

Performance Management Solutions

The latest evolutionary step of BI introduces the concept of corporate performance management (CPM), which is often interchangeably referred to as enterprise performance management (EPM) or business performance management (BPM). CPM is an emerging portfolio of applications and methodology that has evolving BI architectures and tools at its core. Historically, various BI applications have focused on measuring sales, profit, quality, costs, and many other indicators within an enterprise, but CPM goes well beyond these by introducing the concept of "management and feedback". It embraces processes such as planning and forecasting as core tenets of a business strategy. In other words, while the DW process supports the bottom-up extraction of information from data, it does not provide a top-down enforcement of a corporate-wide strategy.

CPM adds a reactive component capable of monitoring time-critical operational processes to allow tactical and operational decision makers to be in tune with the corporate strategy. It also crosses traditional department boundaries or silos to manage the full life cycle of business decision-making, combining business strategy alignment with business planning, forecasting, and modeling capabilities. It is an umbrella term that describes the methodologies, metrics, processes, and systems used to monitor and manage the business performance of an enterprise, whereby applications that enable CPM translate strategically focused information to operational plans and send aggregated results.

These applications are also integrated into many elements of the planning and control cycle, or address BAM or CRM needs. In other words, CPM maps a structured set of data against predefined reports, alerts, dashboards, analysis tools, key performance indicators (KPI), etc., to monitor and improve business processes based on upfront and established corporate strategic objectives. Furthermore, CPM creates a closed-loop process, starting with developing high-level corporate goals and predefined KPIs. It then measures actual results against the KPIs representing the comparison in a balanced scorecard. The results are reported to management through intuitive reporting tools, and are ultimately fed back into the business modeling process for corrections in the next planning cycle.

CPM applications enable information sharing across and even beyond the borders of the enterprise, to all employees, business partners, shareholders, and most importantly, customers. While real time tools can open up BI to tactical and operational decisions, they also amplify the need for an effective information delivery channel. An enterprise portal might be the choice, given its high visibility and it gives companies the opportunity to provide related context, services, and content around published BI information.

Two kinds of BI information seem to be well suited for inclusion in a portal: 1) near real time data, with related content to provide timely snapshots of a business unit or an individual business process; and 2) analysis and summaries, since this high-level BI information fits nicely into executive dashboards, while balanced scorecards can communicate company performance metrics to a broader set of employees, tying it in with budgeting and planning systems, analytical capabilities, and corporate dashboards. Dashboards are front-end presentation that sits on top of CPM systems displaying key metrics and KPIs on which the company wants everyone to focus. Dashboards often represent the window into an overall CPM system, improving the visibility of the results of planning, budgeting, and BI analysis to enterprise users.

Therefore, CPM leverages performance methodologies such as dashboards, balanced scorecards, or activity-based costing (ABC), a cost accounting system that uses cost drivers to allocate costs to products or other business bases to realistically allocate overhead. Although these approaches help determine how and what to measure, they lack a mechanism for dynamically changing values, to keep abreast of the business reality. Ensuring closed-loop management is thus CPMs enhancement of traditional BI applications. BI applications customarily focus on measurement, which is basically worthless without the ability to act upon the results. Consequently, a perplexing variety of existing tools and techniques can lay claim to being part of the CPM trend—ranging from BI tools and analytics to business process management applications (related to but different from BPM), and scorecard products.

Thus, CPM is the evolutionary combination of technology and philosophy, building on the foundation of technology and applications that many enterprises already have. The demand for these applications lies in the fact that they incrementally add value to previously installed business applications, even to legacy ones. With CMP, enterprises may finally see some long belated benefits and feel somewhat better about implementing cumbersome ERP and other enterprise systems. Indeed, many enterprises have already deployed some BI products too, such as querying and reporting tools, planning and budgeting applications, analytic applications, incentive management systems, portals, dashboards, and scorecards, along with data warehousing technology, data models, and integration software, and whatnot. In fact, anyone stocktaking technology inventory will likely find some CPM components already in use.

This concludes Part Three of a seven-part note.

Part One detailed history and current status.

Part Two looked at contemporary BI tools.

Part Four will describe the BI/CPM market landscape.

Part Six will discuss Geac and Point Solutions vendors.

Part Six will compare direct access to a data warehouse for the mid-market.

Part Seven will make recommendations.

About the Authors