An Overview of the Knowledge Based Selection Process

Introduction

The TechnologyEvaluation.Com's (TEC) Knowledge Based Selection Process is a multi-part comprehensive analysis of a vendor's tactical, strategic and qualitative measures as compared to the specific business and technology needs of our clients. The selection process encompasses nine process steps:

Capability Assessment/Request for Information

The first step in the selection process is to identify the capabilities required to support our clients' business and translate those required capabilities into a Request for Information. The RFI is then transmitted to a select list of vendors who have been matched by TEC as a best fit to the specific business needs of our clients.

The goal of the internal capabilities assessment is to establish a set of requirements or criteria that supports the business objectives. An average technology evaluation takes into consideration only a small portion of the necessary criteria needed to adequately evaluate a potential vendor. By and large these criteria focus on functionality and technology and are vital to a potential acquisition, but they are primarily tactical in nature.

At this stage of the process, a TEC specialist walks the project team through an overview of strategic market criteria such as the vendors' ongoing service and support capabilities, their financial viability, and strategy for long term growth and success. Additional qualitative measures are identified to support market feedback from vendor client references, the ease of use and navigation capability of the software, the level to which the vendors proposed solution matches our client's business processes, the vendor diligence level demonstrated throughout the process and the product flexibility as perceived by the client.

This process step allows the project team to concentrate on the totality of the selection factors, not simply those with respect to product features and functions. Vendor RFI responses are validated against market data and entered into the decision-support tool for the initial results.

Bidder's Conference / Vendor Discovery  

After the vendor responses are received and analyzed, the top (typically two or three) scoring vendors are established as the finalists and offered the opportunity to participate in the remaining competitive processes. Each finalist is invited to attend a Bidder's Conference where the client's selection project team members present the unique operational environment and challenges faced to achieve the results of the Capabilities Assessment.

A plant tour is also arranged which provides the first real discovery opportunity for the finalist vendors to learn the specific operating parameters and challenges of the client. Vendors are also offered additional opportunities for discovery to prepare for the Technical Review session and Scripted Scenario demonstration.

Technical Review  

During the scripted scenario preparation period, vendors are required to present their technical solution to a predetermined list of criteria developed collaboratively between TEC and the client. This process step serves to validate specific elements contained within the Product Technology section of the RFI and provides a detailed real time verification of our client's architecture preferences. The areas addressed include:

• General Architecture
  
  
• Degree of Integration among Modules
  
  
• Ability to integrate with other applications
  
  
• Data Protection and Restoration
  
  
• Security Features
  
  
• Tools
  
  
• Performance Degradation
  
  
• Reporting
  
  
• Client Developed Applications
  
  
• Technical Demonstration
  
  
• Migration Path

Client References  

Further sources of information available to the project team are client references. References are contacts in organizations that have previously installed the software and can answer questions about the vendors' implementation services, ongoing maintenance support, and quality of training. In addition, though no two organizations have the same corporate strategy or business requirements, client references can provide useful insights in terms of strengths and weaknesses of the software, challenges they faced during the implementation, organizational impact, and other issues. The project team may visit the references in person or via a teleconference.

Scripted Scenarios  

The Scripted Scenario process step is an opportunity to validate the Product Functionality portion of the RFI through real business activities that need to be supported by the technology. TEC collaborates with our client to develop the scripted scenarios, which are a realization of business processes, current or anticipated, resulting from the Capabilities Assessment. Vendors present these business scenarios on their live products - tailored to the way the organization does business as defined in the scenarios.

These scenarios allow the organization to see how the live product operates in its specific environment, according to the business processes outlined by the project team. In addition, the project team gains an understanding of the extent to which the vendor is willing to adjust the technology for their needs.

The completed scenarios are explained to the finalists' teams during discovery sessions. The vendors are then given a period of time in which to prepare for the scenario demonstrations. TEC develops and provides score sheets to all project team members in attendance for use in rating each vendor's performance. Typically the scenario demonstrations span three days with the scores tabulated daily and entered into the decision support model.

Request for Quotation - Support & Analysis  

TEC provides guidance and support in collaboration with our clients to develop a detailed Request for Quotation (RFQ) for the software and services desired. The RFQ is provided to the finalists (typically at the Bidder's Conference) to allow them to prepare their cost proposal in parallel with the other process activities. This timing provides the opportunity to dovetail all the process elements together to permit a comprehensive analysis of all factors that affect a recommendation without impacting the schedule.

Vendor responses to the RFQ are entered into the decision support model and analyzed. Each finalist is compared relative to one another against the requirements established for our client.

Selection Analysis & Recommendation  

The results collected throughout the process are entered into the decision support model. TEC assists the client project team with the analysis, and facilitation of changes to criteria weights. TEC leads the project team through a sensitivity analysis to understand the impact of weight changes to the criteria on the overall vendor scores. This process is conducted exclusively using the decision-support tool, the trade-off analysis capabilities and additional model variations developed by TEC.

The Weighting Rating Differences Chart is used to show the relative value amongst the factors for one or more vendors. Large values can indicate a likelihood of rank change as the weight is modified in sensitivity analysis. For the expert or consultant, it enables a scale consistency check. In other words, one can check whether the rating given for one vendor against the other for each factor makes sense. Drill down to one or more levels in the model hierarchy is provided to perform more detailed rating comparison and differences evaluations.

Following the collaborative analysis exercise, TEC has sufficient data as well as a clear audit trail of due diligence to provide the recommended selection decision which meets the Capabilities Assessment of the client.

Final Report  

The Final Report captures and presents all data compiled at each process step and serves as the audit document for due diligence. Each data element, RFI, Scripted Scenario etc., are included as appendices to the report. Each appendix item will include vendor responses and scores as appropriate.

Negotiation  

After the recommendation is made, TEC provides the client with guidance and support to put into action any pre-determined counter-proposal or negotiation tactics. These steps may include a combination of the following: a request to lower initial software costs, inclusion of free consulting or training resources, reducing the scope of the services offered, or a decrease in maintenance fees. At this point in the process, TEC provides advice regarding negotiation tactics, possible areas of vendor weakness, and, if desired, can personally negotiate with the vendors. This stage ensures that the organization will receive the highest quality and quantity of vendor products and services at the lowest price.

The TEC Advantage  

TEC offers several distinct advantages in the way that they deliver Knowledge Based Selections:

1. Experienced selection managers. TEC analysts have extensive experience within their given research area, and have lead numerous selection engagements. These individuals have participated in several full cycle implementations, giving them an invaluable hands-on understanding of the marketplace as a whole.
   
   
2. Objectivity. TEC does not participate in the systems integration business or the installation business. This provides a major advantage over other firms such as the Big 5 (Accenture, PWC, D&T, CAPGemini E&Y, KPMG) or boutique-consulting firms who, since they perform integration work, may have a vested interest in which vendor is chosen. Typically, these organizations immediately have available a short list of vendors who happen to comprise their strategic partners. Thus, the decision is often made according to which implementation resources the consulting firm has idle, and not based on what is best for the client.
   
   
3. State-of-the-art decision support tool. Developed in-house, TEC's decision support tool breaks down a decision into its component pieces, and then uses multi-attribute decision science to support a selection decision. This stands in marked contrast to traditional selection decisions that rely solely on gut feel, executive mandate, spreadsheet compilations, etc.
   
   
4. Comprehensive list of criteria. TEC's repositories contain a superset of criteria our clients would want to use in the selection process. Some models contain 10,000 to 11,000 detailed criteria that evaluate not only the quality of the vendors functionality and technology, but also address strategic criteria that indicate whether a vendor will be a reliable partner for the life of the implemented software. These important and often overlooked criteria include the vendor's quality of service and support, financial viability, and corporate strategy. This wealth of criteria ensures that clients do not have to "reinvent the wheel" when drawing up an RFI, RFP or RFQ. In addition, it enables the project team to complete the internal needs assessment in a period of weeks as opposed to months, once again cutting down the time and cost of the selection process.