If you receive errors when attempting to view this white paper, please install the latest version of
Adobe Reader.
"Established in 1993, Technology Evaluation Centers, Inc. (TEC) is the first web-native technology research enterprise. TEC provides decision support systems (DSS)
that enable stakeholders to objectively identify the software products that best fit their company's unique business and systems requirements, and that contribute most effectively to superior business performance. "
Source : Technology Evaluation Centers (TEC)
How to Evaluate Web-based BI Solutions
Web-based BI Solutions is also known as :
choosing web-based bi,
web-based bi,
bi solutions ,
business intelligence software and web reports,
business intelligence solutions,
web-based reporting software,
web based application development ,
build bi solution,
web-based bi resource ,
business intelligence tool,
business intelligence solution evolution,
the business intelligence guide,
bi solution comparison,
web route to business intelligence.
BI on the Fly A Guide to Web-based Business Intelligence
Web-based BI is becoming mainstream. Organizations want to deploy BI
solutions beyond Web portals. This report identifies
emerging trends that make Web-based BI more pervasive within the industry, as
well as the features and functionality to identify essential criteria in
reporting and analysis, online analytical processing
(OLAP), platform, and Web-based development and administration tools.
When selecting solutions in the emerging BI web space, organizations will
evaluate architecture rather than features or functions. This report defines a
framework to evaluate Web-based BI solutions.
This report identifies and rates vendors across the following four areas:
Web-based development and administration tools
Development tools allow IT administrators to develop end user reports, OLAP
cubes, analyses, and performance management tools to allow end users to leverage
BI tools.
OLAP capabilities
OLAP allows users to analyze data using
dimensions and drilling through information to create different views to capture
the required data. Generally, OLAP cubes are relegated to super users. However,
they can be used to interact with reports and provide users with the ability to
create customized reports by developing their cube views based on their business
needs.
Platform ' technology and administration capabilities
Web-based platforms allow both administrators and end users to access and to
modify BI requirements within a purely Web-based environment. Thus, the BI
platform can be accessed via the Web, enabling access from the Internet without
having to access an Intranet or internal network.
Reporting and analysis
In many cases, reporting and
analysis represents the bulk of BI tools that are used by organizations.
Organizations require the flexibility to analyze data in multiple ways, to
change reporting parameters, to update data in real time to help drive business
decisions, and to manage reports based on user responsibility and collaboration
among teams. Reporting and analysis includes both pre-canned and adhoc reporting capabilities.
Vendors represented are Business Objects and their Crystal Reporting
products, Information Builders' WebFOCUS, LogiXML's Logi 8, and Microsoft's BI
offerings deployed via SharePoint. Each offers varying degrees of Web-based
reporting and analysis, with LogiXML being the only fully Web-based BI suite.
Defining the Web-based Business Intelligence Space
Web-based BI is becoming more important within the BI industry, as
organizations are looking for less invasive, and less expensive ways to benefit
from reporting and analytics. Although most organizations implement traditional
BI applications, use of the Internet to collaborate and to deliver reporting and
OLAP is already a main element of software deployments. As the demand for Web
2.0, software as a service (SaaS),
service-oriented architecture (SOA), and hosted
applications has increased, organizations are increasingly examining Web-based
applications.
There are two approaches to Web-based BI delivery. The first involves portal
delivery of BI reports and OLAP cubes, and the second is a true Web-based BI
solution, fully deployed and maintained via the Web. From the user's
perspective, the differences may seem minor, as the interface remains the same.
The real difference is the back end and the fact that development and
maintenance are Web-based. Key features of Web-based BI include the following:
- Delivery
Organizations are able to deliver and to
standardize their BI environments over the Internet through either approach.
Hence, reporting, OLAP, and analysis are provided to users within a
Web-based interface using a centralized platform.
- Development environment
Web-based BI provides a
centralized platform that is managed over the Web. Thus, organizations may
reduce the need for (or have less concern about) server space and the
management within a client server environment.
- Licensing structure
Users may take advantage of more
generous licensing structures based on processors or applications, as
opposed to client and server architectures requiring limited user licenses
that drive up software costs.
- Collaboration
Web-based applications provide users
with a single point of contact for reporting, analysis, OLAP, and
collaboration.
Web-based Business Intelligence Market Drivers
Web-based BI market drivers include user demand for the following:
- Seamless distribution and upgrades
Web-based BI
solutions are upgraded easily and do not require downloads on user terminals
to remain current.
- Ease of use
Web-based BI solutions leverage the
intuitive nature of the Internet. Additionally, consistent user interfaces
shorten the learning curve.
- Ubiquitous access
Users can access information in an
interactive, dynamic, and usable form from any location.
Vendor Overview
The building and marketing of purely Web-based BI solutions are not as common
as fully integrated BI offerings. In many cases, vendors are providing a portal
approach as opposed to a Web-based approach. Many of these solutions are based
on integration with Web services and SOA. For the purposes of this report, these
solutions are placed under the umbrella of Web-based BI solutions because, for
users, the interfaces are the same.
Business Objects
Business Objects provides BI solutions for both large enterprises and
small to medium businesses (SMBs). BusinessObjects&8482;
XI is a complete solution that includes performance management, reporting,
query, and analysis, and enterprise information management. Business Objects
delivers Web-based reporting online through Crystal Reports, which can be
integrated with its BI platform or used as a stand-alone reporting tool. With
its acquisition of NSite Software, Business Objects intends to add SaaS-based BI
to its product offerings, creating more choices for current and future clients.
Although Crystal Reports can be deployed via the Web and integrated with other
BI infrastructures, a full-scale implementation currently involves commitment
from IT and the business units to install and to deploy BI throughout the
organization. Business Objects is one of the largest, pure-play BI vendors,
generating more than $1 billion (USD) in revenue.
Business Objects' Crystal Reports is a strong Web-based reporting solution.
Without an OLAP server and the ability to write data back to a database, reports
can be generated and rerun, but cannot be updated until data has been reloaded
into the database and rerun. Organizations that require minimal data latency and
automatic updates should look at other options to fulfill those needs. Business
Objects' move toward SaaS-based BI solution integration and targeted offerings
for SMBs gives its customers multiple options for deployment.
Information Builders
Information Builders WebFOCUS provides a fully integrated suite with
Web-based reporting features that allow access to reporting, analysis, and OLAP.
The vendor's products include an active metadata layer and full integration
services through its iWay Software. WebFOCUS uses an SOA that supports Web-based
delivery of its solutions, as well as integration with Web-based applications
already in use within the organization. Information Builders also provides
options for hosted reporting solutions to enable flexible deployment options for
users to analyze their mainframe and Unix-based environments. WebFOCUS targets
the enterprise market and real-time, operational reporting area.
Information Builders' WebFOCUS offers Web-based and hosted reporting
solutions based on an SOA platform and integration with Web services. Its back
end is client'server based, but its Web-based integration can provide
organizations with easy integration within Web-enabled environments. However,
the lack of an OLAP server means that data cannot be written back to the server
or updated to reflect real-time updates within that feature.
LogiXML
LogiXML delivers a comprehensive, Web-based BI offering. The product's main
components are Web-based reporting and ad hoc analysis capabilities.
Additionally, LogiXML provides a Web-based development environment, enabling
easy implementation through downloads as opposed to organization-wide
integration activities. Unlike many other BI vendors that provide a Web front
end only, LogiXML's offering also provides a back-end platform and development
tool set that are Web-based. Thus, the vendor's delivery is a unified approach
for organizations within the mid-market that want easy deployments and
management of their BI environments without the added integration and management
issues.
LogiXML provides an overall Web-based solution that enables easy upgrades and
implementations. The solution's functionality is comparable to the solutions of
the other vendors evaluated here, with the main differentiator being its
Web-based development environment. The key advantage of a full Web-based
solution is that it allows downloadable updates that enable seamless upgrades
and maintenance. Aside from the ability to develop reports within a Web-based
environment, the ability to upgrade software versions without the hassle of
integration with current IT environments can save organizations time and money.
Microsoft
Microsoft's BI offerings include Business Scorecard
Manager (BSM), the SQL Server 2005 data warehouse, analysis services, and
reporting services. Organizations may purchase each product separately, as
needed, or together to create a full BI offering. Additionally, Excel 2007 users
can move beyond general spreadsheet functionality because of the application's
integrated BI capabilities. With the acquisition of ProClarity in 2006 and the
release of BSM, Microsoft has created a natural inroad into the BI market,
thereby better positioning itself for SMBs.
Microsoft has built its BI solutions to be fully integrated with its MS
Office offerings and to be deployed through SharePoint for Web-based access.
Microsoft's mid-market focus and current market penetration based on current
market saturation makes the vendor a strong contender within the mid-market.
Through SharePoint, Microsoft's BI provides strong integration features for
portal-based BI and other applications.
Methodology
The evaluation criteria are separated into four areas: Web-based development
tools, OLAP, platform, and reporting and analysis. The ratings are based on
criteria identified as key factors for organizations to consider when evaluating
a Web-based BI tool. The ratings are shown in the chart below, and scoring
relates to the level of support vendors provide. Organizations should use these
criteria as a guide, but should also realize that a needs assessment should be
completed, and the results measured against the ratings identified to select the
vendor offerings that best meet the organization's BI requirements.
Rating Definitions
The ratings below identify the scoring criteria for the vendors based on the
four criteria listed above:
Rating Component Definitions
The report evaluates the vendor products across four categories, representing
sixty criteria in total. The chosen criteria reflect features and functionality
that organizations should consider when evaluating Web-based BI solutions
Web-based Development and Administration Tools
Web-based development tools allow IT administrators to develop user reports,
OLAP cubes, analyses, and performance management tools. The advantages of
Web-based development include ease of use, Web-based creation, and management of
reporting and analysis environments.
- Drag-and-drop environment
The ability to develop
cubes and reports within a drag-and-drop environment without having to use
programming languages to develop these cubes and reports
- Reusable elements (heat maps, animated charts, graphs, etc.)
Elements can be reused across development environments
- Graphical interface that avoids complex languages such as C or
Java
Developers can develop front-end applications using wizards
or through drag-and-drop
- Wizards for synchronizing tables and database objects
Databases and subsequent tables can be linked or updated using wizards that
allow users to choose the database tables and associated objects
- Synchronization of source system with metadatabase tables
Allows metadata to be managed, and record and table relationships to be
maintained over time
- Electronic distribution of any content via preferred delivery
vehicles, including e-mail, file transfer protocol
(FTP), or network printers and faxes
The ability to
distribute reports, cubes, and so forth using different distribution methods
- Task and report scheduling to reflect daily, weekly, or monthly
delivery
The ability to schedule tasks and reports to be
delivered at specific time frames
- Security tailored to meet user- and group-based roles
Access to reports and analytical tools can be managed based on roles and
groups defined by the administrator to limit access to security-based roles
- Ability to create, to manage, and to customize virtual table
views
Data can be pooled to provide instantaneous views of data
virtually
- User interface customization via cascading
style sheets (CSS)
Developers can customize user
interfaces via Web-based standards within a Web environment
- Alerts, triggers, and notifications to allow for collaboration
The ability to collaborate across user groups through scheduling or
automated alerts and notifications based on triggers or identified processes
OLAP Capabilities
OLAP allows users to analyze data by using dimensions and drilling through
information to create different views to capture the required data.
- Reads compact reports through hypertext
transfer protocol (HTTP) and file transfer protocol (FTP)
The ability to view reports via Web-based protocols
- Integrates into any Web application
The ability to
integrate cube views into Web applications internally or externally
- Creates personalized cube views
The ability to
create OLAP views that are customized to the data that users want to view,
in the way that they want to view it
- Performs calculations based on date and time, such as trend,
variance, and cumulative sum functions
The ability to identify
trends and associated calculations over time
- Calculates facts based on predefined functions or user-defined
formulas
The ability for users to develop their own algorithms
and formulas based on specific needs
- Updates, synchronizes, and refreshes cube views for real-time
analysis
The ability to update cube views based on real-time
access to and updates from connected data sources
- Exports to other formats for analysis (XLS, PDF, RFT, CSV, etc.)
Cube views can be exported to external sources to perform more in-depth
analyses based on user preferences
- Enables interactive drilldown into multiple detail levels
The ability to drill through dimensions to view data in more (or less)
detail
- Drag-and-drop functionality
Allows users to drag and
to reorder dimensions, and to create different cube views
- Displays tree structure views of data dimensions and measures
Allows users to view the hierarchy of data dimensions and information
available to create and to customize cube views
Platform Technology and Administration Capabilities
Web-based platforms enable both administrators and users to access and to
modify BI requirements.
- Fully Web-based platform (no ActiveX or Smart Clients)
The platform is purely Web-based, without customization or modification
requirements
- Extensible markup language (XML)-based
criteria as a data source and as an export
The ability to use
XML-based environments as criteria to populate data sources and to export to
various applications
- Built-in report styles
General reporting styles and
types built-in to allow for easy report generation
- Data integration from the Web
The ability to
integrate Web-based data into database environments or reports
- Metadata displayed in both technical and business views
Users can view metadata in formats other than relationship-based tables
- Platform-independent architecture that integrates with existing
infrastructure
Allows the current environment to interface and
to integrate with the Web-based structure
- Options to update or to synchronize data from multiple sources
via automatic schedulers, or on demand
The use of schedulers or
on demand features to update data
- Report back-up and archiving
Activities performed
can be backed up and archived based on scheduling or requested functions
- HTML generation of project documentation, allowing projects to
be posted on intranets for review
The ability to generate
HTML-based documentation to share with users
- Creation of activity logs to view user activity
The
ability to monitor user activity to identify changes and views to data,
reports, and OLAP cubes, and to account for regulatory requirements
- Data modeling and process modeling integration within the same
tool set to share data definitions and to reuse data sets
The
ability to reuse business rules and reports based on created data models
- Saves commonly used queries for quick retrieval and execution
Queries can be reused for report and analysis execution
- Ability to interface with Web services
Back end
operations can interface with Web services
Reporting and Analysis
In many cases, reporting and analysis represents the bulk of BI tools in use.
Organizations require the flexibility to analyze data in multiple ways; to
change reporting parameters; to update data in real time to help drive business
decisions; and to manage reports based on user responsibility and collaboration
among teams. Reporting and analysis includes both pre-canned and ad hoc
reporting capabilities.
- Real-time access to operational data stores
The
ability to connect to various databases and data stores to access and to
pull operational data to load reports in real time
- Preset report templates
Reporting templates that are
built into the software to provide users with pre-canned reports
- Report wizards to guide users through analytical and reporting
tasks
Wizards allow users to develop reports through a
step-by-step function
- Wizard functionality to create simple or compound metrics
Metric creation allows users to tie their analyses to performance indicators
based on operational data and organizational goals
- Ability to drill down on or to hide detailed information
Users can drill through to more detailed representations of data or hide
data based on analysis or user specifications
- Modifiable report templates
Allows users to modify
and to change report templates to reflect individual or business unit needs
- Automatic drill through to several reports from a single page
The ability to access several different reports through one report to access
a subset or different view of data based on the drill path
- Access to analytics from Microsoft Office applications
Interfaces with Microsoft Office applications, such as Excel
- Nested crosstabs and charts
Crosstabs and charts are
embedded within the reports to enable a more complete view of data
- Creation of analytic workflows
Allows users to
develop a process map and to link it to their reports
- Report prompts to give multiple selection criteria
The ability to customize reports based on parameters
- Add calculations, such as sum, average, moving average, and
count to reports
The ability to customize at the user level
different calculations and totals based on user needs
- Changes graphical views in real-time
Updates
information and changes views of data automatically to reflect changes in
data
- Exception highlighting to identify problem areas within a path
of data in real-time
Sets thresholds for data or metrics, and
notifies users through exception reports
- Decision analysis and decision tree display
The
display of process flows and information hierarchies through graphical flow
charts
- Distributes content across the enterprise via the Web
The ability to share reports across the Web to enable collaboration within
the organization
- Uses portals to deliver customized information enterprise-wide
The delivery of reports to various users across the organization
- Sets qualifications based on a relationship filter or metric
Identifies and pulls specific data based on filters or defined metrics
- Creates custom expressions
The ability to customize
report calculations
- Ability to write and to save data to the database
Users can change report data and write that data back to the database to
update incorrect or out-of-date information
Results
The graph below displays the overall vendor ratings based on the sixty
criteria, and how vendors rate in comparison with one another within a Web-based
environment. The scores reflect the overall average of vendor totals for all
four sections Overall ratings do not provide a full view of vendor capabilities.
For organizations to identify the appropriate solution, a deeper analysis is
required. We will now drill down from the overall scores and examine each of the
four categories. In the sections that follow, we will provide two graphs per
section. The first presents the vendors' overall ratings for the category, and
the second shows the vendors' levels of support within the category, indicated
as follows: SUP, MOD, CST, 3RD, FUT, or NS.
1.0 Web-based Development Tools and Administration Overview Graph
The overall Web-based development and administration tools ratings graph
identifies the vendors' offerings based on their Web-based development
capabilities. Key functionality includes a Web-based development environment,
synchronization of source system with meta-database tables, element reuse, and
wizard-driven environments to synchronize tables and database objects.
The chart below provides a breakdown of the totals for Web-based development
criteria. The main rating differentiation occurs based on whether the vendor
provides full Web-based BI solutions or portal-based delivery of reporting and
analysis applications
2.0 OLAP Capabilities Graph
The overall OLAP ratings identify the vendors' abilities to offer OLAP
features and functionality in comparison with one another. Key functionality
includes the ability to update, to synchronize, and to refresh cube views for
real time analysis, integration with Web applications, date- and time-based
calculations, and personalized cube views
The ratings below represent the criteria reflected in the totals above. The
key difference in scores results from vendors' respective use of OLAP servers to
update and to write data back to the database in real time.
3.0 Platform Technology and Administration Capabilities Overview Graph
The following graph identifies the overall ratings for each vendor's platform
capabilities, including their technology and administration capabilities. Each
vendor's platform strategy differs.
The chart below identifies the ratings for the platform criteria. For the
purposes of this report, those vendors with greater levels of platform support
scored higher. However, user organizations have unique platform requirements and
strategies, and thus should choose vendors based on best fit, not overall
ratings
4.0 Reporting and Analysis Overview Graph
The following graph identifies each vendor's total reporting and analysis
ratings. Reporting and analysis is a relatively mature functional area.
Consequently, most vendors provide these features fully supported out of the
box.
Conclusion
From the analysis presented here, two categories emerge as key
differentiators among Web-based BI vendors: Web-based development and
administration tools, and OLAP server capabilities. The ability to develop, to
deploy, to maintain, and to upgrade offerings within a Web-based environment
speeds implementation times and eliminates the need for integration with
organizational IT environments. As for OLAP server capabilities, vendors that
offer this capability enable users to update data in real time, allowing for
dynamic analysis of data. For organizations that require immediate data updates,
this type of solution will be more beneficial.
The evaluation criteria represent standard features and functionality that
should be deployed as part of a full-scale BI solution. Although certain
criteria may not be offered by all vendors, organizations should evaluate
whether the missing criteria are important to the specific solution they need.
Also, organizations should evaluate vendor positioning and strategy related to
Web-based offerings. Lastly, organizations should consider their BI objectives,
their current and future use of BI, and their overall Web strategies before
starting the selection process.
by Lyndsay Wise
May 3, 2007