Originally published - January 10, 2005
Business intelligence (BI) is a big buzzword in today's IT world. It is a concept understood by some, but misunderstood by many.
The purpose of this article is to clarify what BI is and to discuss its main components. It will also discuss why BI is drawing so much attention, even during this economic downturn, and why it is not only winning the hearts and minds of decision makers, but is also penetrating many functional areas like marketing, finance, and securities.
Part of the confusion about BI lies in the flurry of acronyms relating to analyzing business information. In addition to business intelligence, terms like business performance management (BPM), business process management (also BPM), corporate performance management (CPM), and business activity monitoring (BAM), have also emerged. All of these are a part of BI. They are all dependent on BI tools, but it should be noted that BI is not dependent on them.
Though the name was coined by the Gartner Group in mid nineties, business intelligence as a concept started much earlier. The concept was rooted in the reporting systems of mainframe computers in the seventies. During that period, reporting systems were static, two-dimensional, and without analytical capabilities. The demand for dynamic multidimensional reporting systems for predictive and intelligent decision-making pushed BI to develop. With the advent of new technologies and applications, BI came to its present state and is continuing to grow every day. BI today is capable of multidimensional analysis of data to see 360 degree business insight, statistical analysis, and forecasting to help better decision support systems.
Based on present trends, the use of BI will become so widespread that every desktop will have a BI icon that in the future. BI will become an integral part of an enterprise's information system and, like word processing software, BI will be used by almost all end users, business users, and government officials to gauge whether their strategies are aligned with their companies' overall strategic plan.
What BI Is All About
BI is neither a product nor a system. It is an umbrella term that combines architectures, applications, and databases. It enables the real-time, interactive access, analysis, and manipulation of information, which provides the business community with easy access to business data. BI analyzes historical data—the data businesses generate through transactions or by other kinds of business activities—and helps businesses by analyzing the past and present business situations and performances. By giving this valuable insight, BI helps decision makers make more informed decisions and supplies end users with critical business information on their customers or partners, including information on behaviors and trends.
Businesses generate a sea of data. Every datum carries a small piece of the business' story. This data is scattered everywhere, in disparate systems and in different departments. It is held captive in dead hard drives, and can even be situated in geographically different regions. However, it is in data, where the true nature of business—its trends, strengths, and weaknesses—lie. BI gathers all the related data to turn it into information and information that is analyzed properly can be used for decision making which can finally go into action.
In other words, BI transforms data into information, information into decisions, and decisions into action.
How BI Evolved
Key to understanding how BI analyzes business is understanding how data is processed into information (via different technologies) and how it is analyzed. Knowing these processes and how it fits into BI's architecture, tools, and applications will also provide clarity about BI.
BI doesn't produce any data, but it uses data produced by other business applications like enterprise resource planning (ERP), customer relationship management (CRM), supply chain management (SCM) etc. Over the past two decades, and especially in nineties, organizations have stored huge amounts of data by building online transaction processing (OLTP) systems and ERP systems, call centers, and the Internet. In pursuit of better data management enterprises build data warehouses (DW), data marts, and installed extract transform load (ETL) tools to work with data warehouses. But very few of these data were processed into information, and even less was used for decision support systems, largely because of the lack of tools to access and analyze the data for business users.
In seventies and eighties, accessing information systems was very tedious and it was rarely permitted to end users. Query and reporting was cumbersome and analytical reporting was spreadsheet based. The whole process of accessing information was time consuming, and delayed reporting didn't achieve results. The advent of technology and the increasing demand from companies to have better information, pushed BI systems to evolve.
BI has a tremendous impact on business once installed. It produces the right information at the right time, which is key element for the success of any business enterprise. BI is the art of knowing and gaining the business advantage from data. Whether it is marketing competition, customer retention, inventory control, financial modeling, or even in national security, BI is the answer. BI can answer a company's critical questions such as, why market shares are going to competitors; which products contribute the most to profit; how can business become more profitable; why some divisions are not profitable; which plants produce at the lowest cost; how can productivity improve; which parts of the world are the most profitable; who are best and worst customers; where is money being lost or made, etc.
BI answers these questions by analyzing and comparing business historical data. Data is created by business activities or data from outside sources like environmental, demographic, immigration data, etc. to study a particular group of people or customers. Such information is used by businesses to understand their business trends, their strengths and weakness, and to analyze competitors and the market situation. The information can also be used by government or secret agencies, especially like US Homeland Security, that need to have access to financial, immigration, transportation, and any kind of related data that can be analyzed to determine probable attacks on its citizens or property.
In addition to determining trends, another push to implement BI comes from, the Sarbanes-Oxley legislation, which affects corporate financial reporting, and accounting rules for publicly-held companies. To be in compliance with Sarbanes-Oxley, BI systems will be needed to insure the timely and accurate analysis of business data. Thus real time BI is not only relevant but key to achieving compliance.
Business Intelligence Activities and Tools
As mentioned earlier, BI is a combination of technologies and architectures. Some important BI tools are data warehouses (DW) and data mart, extract transfer load (ETL), reporting and query tools, data visualization, balanced scorecards, dashboards, OLTP, OLAP, data mining, alerting and notification systems, and analytics. Under the BI umbrella all these tools are combined within a special architecture.
Data Warehouses and Data Mart
Because data can come from various sources, like OLTP, ERP, CRM, legacy applications, and external data sources, data can be stored in a diversified database, in different formats and structures. As a result, a data warehouse (DW), is the most important and expensive player in the whole BI system because it captures data from these diverse sources, and unifies them. The data is then ready to be accessed by BI system. As a central repository of business, DW contains data used for decision support systems (DSS) which focuses on the lower and middle management and makes it possible to look at and analyze data in different ways. Such data also used for executive information systems (EIS). Data extracted from the DW by departments are collected and put into smaller repository for easy and fast access are called data mart. Like data mart for marketing data, sales, production etc.
Extract Transfer Load (ETL)
The process of populating data into data warehouse is done through extract transfer load or ETL, which is a set of three separate functions—extract, transfer, and load. First, the extract function reads data from a specified source and extracts a desired subset of data. Next, the transform function works with the acquired data—using rules or lookup tables, or creating combinations with other data—to convert the extracted information to the desired state. Finally, the load function is used to write the resulting data to a target database, conversion from one database type to another, and the migration of data from one database or platform to another.
Sometimes DW and BI functions are confused. The difference between the two is that DW does not require BI for functionality. For example, reports can be generated from DW through reporting tools. However, the vice versa is not true. BI needs DW to access accurate and selective data, however, there are exceptions to that. In recent days some vendors designed their architecture to access data directly from the source as opposed to DW. Also, cost-wise, BI is more expensive, but with BI architecture, there is flexibility to select from different tools. The high cost of BI can be offset by companies if they take advantage of building BI infrastructure using tools they already have installed.
Reporting and Query Tools
Reporting is the process of accessing data, formatting it, and delivering it as information. Report and query enables users to issue structured query language (SQL) queries to the warehouse. It allows users to view and report specific information they require. Reporting is one of the main functions of BI.
Data visualization tools help users to see data clearly. It is the graphical representations of data, including complex three-dimensional data pictures. Data visualization tools interpret information and data relationships. It combines representations of multiple data sets simultaneously, or gives multiple views of a single data set, even a set that encompasses millions of data points. Tools include of data visualization include charts and graphs, dashboards, and scorecards.
Balanced scorecards (BSC) are a concept that help users translate strategy into action. It is a performance measurement system, derived from vision and strategy, and reflects the most important aspects of the business, such as customer knowledge, financial performance, internal business processes, etc. BSC is a central list of pre-defined numbers, which show each key part of an organization's success. BSC focus on strategic views of management goals, and then hold people accountable for what management has set. Through BSC, managers at all levels monitor results in their key business areas.
A BI dashboard is a user interface that somewhat resembles an automobile's dashboard and organizes and presents information in a way that is easy to read. However, a computer dashboard is more likely to be interactive than an automobile dashboard. To some extent, most graphical user interfaces (GUI) resemble a dashboard.
As mentioned earlier, companies store vast amounts of information (largely operational data) on on-line transaction processing (OLTP). OLTP data is always on-line, and is fast and efficient for querying. It facilitates and manages transaction-oriented applications. OLTP enables analysts, managers, and executives to gain insight into data through fast, consistent, interactive access to a wide variety of possible views of information. Probably the most widely installed OLTP product is IBM's CICS (Customer Information Control System). OLTP data is stored in data warehouse.
OLAP or online analytical processing is another component of BI. It enables a user to easily and selectively extract and view data from different points-of-view. OLAP data is stored in a multidimensional database. The main component of OLAP is the OLAP server, which sits between client and database management systems (DBMS). OLAP tools enable users to analyze different dimensions of multidimensional data. For example, it provides time series and trend analysis views. OLAP often is used in data mining.
Data mining tools automatically extract hidden, predictive information from databases. It also searches for the patterns in large transaction databases. These tools are usually driven by complex statistical formulas. The easiest way to distinguish data mining from the various forms of OLAP is that, OLAP can only answer questions you know to ask, data mining answers questions you didn't necessarily know to ask.
Alerting and notification systems
Alerting and notification systems proactively deliver business intelligence to users based on criteria they define, as the events happen. It keeps users continuously informed of critical events by delivering the message at work, home, or on the road through e-mail and wireless technologies.
BI versus Analytics
Business intelligence is sometimes called analytics though there are distinctions between them. BI is full set of technologies and programs and analytics comprises all specialized programming that analyzes data about a particular field, like marketing, sales, real estates etc. which helps take better and quicker business decisions. Presently there are many analytics available in the market such as CRM analytics, supply chain analytics, sales analytics, customer analytics, product and service analytics, finance analytics, real estate analytics, and Homeland Security Analytics, etc.
Selecting Right BI Software
BI has captured the attention of many organizations for it's intelligence about the business, but to maximize its power, you cannot buy a BI product off the shelf because it is not a one-size-fits-all system. Every business is unique and business processes and business rules differ. BI should be customized to meet the unique features of an organization.
Recommendations for BI application should be based on a company's functional requirements, budget, technical architecture, and overall user need. Selecting and implementing the right BI is a challenging job. Implementing BI is a costly and time consuming venture. If the wrong BI is implemented without good research and planning it could be a failure initiative. One very important point to be considered for selecting BI is there should be a close match between company's requirements and vendors provided solutions. Companies seeking to implement a BI system should make sure they research their needs thoroughly. When looking for consultants, make sure they have detailed knowledge bases of vendors that include functional RFIs.
In today's highly competitive business, the quality and timeliness of business information for an organization is not the choice between profit and loss, it is a question of survival or bankruptcy. No business organization can deny the inevitable benefits of BI. Recent industry analyst reports show that in the coming years, millions of people will use BI visual tools and analytics every day. Visualization tools will be used by producers, retailers, government and special agencies. More and more industry specific Analytical tools will flood the market to do any kind of analysis and help to make informed decision making from top level to user level.
Another potential trend involving BI is its possible merger with artificial intelligence (AI). AI has been used in business applications since eighties, and it is widely used for complex problem-solving and decision-support techniques in real-time business applications. It will not be long before AI applications are merged with BI, bringing in a new era in business. It's also not unlikely that one day BI may adopt new name of artificial business intelligence (ABI).
Business intelligence is spreading it's wings to cover everyone, from small, medium, and to large companies. As large BI players are for large enterprises, small, niche players service mid-size or small companies. Analytics tools also penetrating into the market for very specialized functions, which will help some companies to go for analytics instead of full BI implementation. Moreover, the argument to replacing traditional BI with analytic applications is not an immediate threat. According to industry analysts, this will take years that to happen.
BI takes the advantage of already developed and installed components of IT technologies to helps companies leverage their current IT investments and use valuable data stored in legacy and transactional systems. For many large-size companies that have already spent millions of dollars building data warehouse and data marts, now is the right time to build BI as next step to get full benefit of their investment which will directly impact return in investment (ROI).