A Case Study and Tutorial in Using IT Knowledge Based Tools Part 1: Decision Support Discussion

Part 1: Decision Support Discussion
E. Robins - May 30, 2001

Executive Summary   

Most business managers, whether vendors, vendor clients or implementers, are unaware of the fundamental capabilities that knowledge based decision support can provide to minimize project risk for all sides of technology utilization. Given that over 90% of IT projects fail on first attempt, according to the Standish Group, more thought and research on the evaluation and selection process is needed. Many of these failures - some 30% - are the direct result of poor selections, and represent upward of $30 billion in wasted investment annually.

On the vendor side, the challenge of educating the potential client of their offerings results in long sales cycles, meticulous and numerous RFI responses, and potential for a mismatch, resulting in projects that can go awry. These failed projects do not bode well for the vendor, since the sales cycle costs can only rise, and their reputation can suffer. Consequences can be more severe for the client where it can, in extreme cases, lead to business failure. For implementers, the issue is similar: having inadequate information for the implementation phase means an inability to properly plan and execute the implementation, or for a consultant to assist the end client in making proper technology decisions. Implementers (which can be internal IT departments as well as consultants) can also find that decision-maker indecision leads to lengthened sales cycles, missed opportunities, and risk of competitive intrusion. The root cause of this indecision is an inability of the implementer to give confidence to the stakeholders of their choice of solution.

All sides need thought and research to build data and process information in a meaningful context, which takes time and costs money for all participants going through the selection process. But without spending time, thought, research and money there is increased business risk to all.

To cut away from this devil-and-the-deep-blue-sea conundrum means looking under the hood of evaluation and selection practices, to determine if there are better ways of moving through them. There is certainly room to ask the fundamental question of whether the current practice of RFI / RFP processes, among other internal organizational procedures, are adequate to the task of selecting complex systems. The record indicates there is much room for improvement.

In essence, for complex selections, the human-machine combination has to work together to drive the solution. Both have to be understood and complement each other in the process. It is easy for the human to be overwhelmed, or simply run out of time, and the machine interface and engine to be inadequate to the task. However, the results must benefit the process if human and machine can function effectively together to process information and avoid the pitfalls of past selection processes.

In the second part of this article, we shall follow a simplified process as an illustration. This method was used by the author to conduct the selection on a personal device assistant (PDA). Though a PDA is far less complex than an ERP system, processes and procedures enabling narrowing down of solutions, and avoiding dissatisfaction, while taking on assessed risks, are part of the process embedded in Knowledge Based Selection methods.

About This Note: This is a two part note with Part 1 containing a discussion of the use of an IT Knowledge Based selection tool as part of a Decision Support System selection process. Part 2 is a tutorial which illustrates using such a system to select the personal device assistant (PDA).

Overview of Decision Support System   

Traditionally, DSSs (Decision Support Systems) have largely been used for internal corporate support. However, there is a growing trend to combine DSS and knowledge bases for product and project evaluation that can benefit all sides, leading to a methodology called Knowledge Based Selection. Getting the methods and the technology right is important. This is the main focus of TECs value proposition through its research and tools development programs.

Maximizing the benefits from using knowledge driven selection processes requires two key components: accurate data and a clear process enabling stakeholders to navigate through the data to get to a solution. If the system provides these capabilities, the benefits can include:

1. Narrowing products down to a shortlist. Vendors benefit from not pursuing unfruitful clients, and clients benefit because the short list is usually reduced to a manageable size
   
   
2. Entering the end client business scenario early on enables a better fit and more rapid narrowing of product match
   
   
3. Identification of issues and negotiation perspectives are brought to the front, enabling more efficient and productive negotiations
   
   
4. Enabling the construction of scenarios and measuring scenario performance to reach the final decision
   
   
5. Building confidence in the decision among the client team
   
   
6. Building buy-in to a decision at the client site, easing political obstacles to the selection
   
   
7. Enabling the solution implementers to be better aware of the challenges
   
   
8. Enabling vendors to be aware of product gaps with client needs
   
   
9. Enabling expectations of the implementation results to be realistic
   
   
10. Enable better implementation planning
    
    
11. Creating a more reliable and realistic outcome
    
    
12. Enable future project discussions between the vendor and client to be processed more effectively, since past data is intact and in a form that is reusable and can be updated easily.

The result of the exercise is a decision in which the business risk is minimized and stakeholders have reached consensus, and mutual understanding in the minimum time. Implementers are more thoroughly aware of the issues they face, and the vendor has a better customer relationship as a result, leading to potentially more future business.

Knowledge Based Selection is a Tool   

The last thing a knowledge based selection method does is to make the decision for you. It is a tool that is part of a process, not the complete process. However, it can be a very revealing component essential in the overall quality of the result. The method consists of ways to rationally input information, and then to evaluate the information according to the value requirements of the stakeholders. Hence a component in the mix is a knowledge base to store information, and an evaluation engine then enables the stakeholders to navigate through it and estimate each solution's value and risk in the context of stakeholder requirements.

Intrinsic processes within the evaluation engine must enable at least five things:

1. Enable the inclusion of the value proposition to the end user
   
   
2. Enable a narrowing down to a few solutions with high probability of delivering core technology
   
   
3. Highlight the differences among the lead solutions to better understand the business tradeoffs that may have to be made
   
   
4. Enable a better and deeper understanding of the selected solution
   
   
5. Give confidence that the selected solution will meet at least minimum needs

Over the past few years, many decision support tools and systems (DSSs) have appeared on the Internet. In IT, most of these are based on 'value trees'. Value trees are intended to accurately measure the degree of worth of a solution to a business case, and if done correctly, will reflect the value proposition of a solution to the required solution. From the science perspective, most online DSS systems are simple and rarely provide the insight needed in making major decisions, suffering from at least one of five major flaws in IT solutions selection:

1. Methodologies do not adequately represent the value propositions that need to be met for each business case.
   
   
2. The depth of information (content) is inadequate.
   
   
3. The processing is done from a features and functions level, not a business objectives and required capabilities level
   
   
4. The processes to uncover and make sense of content are inadequate.
   
   
5. Vendor client match methods lacks appropriate internal processes. This often leaves many good niche players swamped by larger organizations who can claim they 'do everything': niche players do some things very well, and generally should be considered in particular business scenarios.

In essence, the human-machine combination has to work together to drive the solution. Both have to be understood and complement each other in the process. It is easy for the human to be overwhelmed, or simply run out of time, and the machine interface and engine to be inadequate to the task. However, the results must benefit the process if human and machine can function effectively together to process information and avoid the pitfalls of past selection processes.

This concludes Part 1 of a two-part discussion of using an IT Knowledge Based tool in a Decision Support Process. Part 2 is a tutorial, which illustratively discusses using such a tool to select a personal device assistant (PDA).