In part one of this series, we covered the first phase of a best-practice software selection project, including the four steps involved in the definition of business and technical requirements. We'll turn now to the all-important final phases of the enterprise software selection process.
Phase 2: How to Assess and Evaluate Software
The output from Phase 1 is a detailed list of functional requirements. Phase 2 covers the assessment and evaluation of all available solutions on the market. Solutions should be compared on a level playing field so that decisions can be made intelligently and fairly. This also means that all the vendors you're considering can clearly understand the functionality required and respond accordingly. The following steps will describe Technology Evaluation Centers' (TEC's) methodology for software assessment and evaluation, and the ideal way of selecting two or three of the best solutions for the final assessment. At this point, a selection team should be appointed by management, comprised of key members of the project team.
Step 1: Use Prequalification to Establish Your Long List of Vendors
The selection team is now tasked with the complicated process of finding which vendors can support the requirements of your proposed system. Much research is required to be able to identify which vendors can support most of the functionality, and whether more than one vendor will eventually be needed.
This is a time-consuming part of the process, and some key vendors may be overlooked if due diligence is not exercised. It's easy to identify the main players in any given area, but there are many excellent vendors that are not household names in the market. Normally, there are at least 15 to 20 vendors that may qualify as a viable solution provider, and as many as 50 vendors in some cases. It's nearly impossible for an organization to research all these without professional help.
Based on your organizational attributes and high-level requirements, TEC's decision support tool generates a list of pre-qualified vendors that should be included in your long list. The system is flexible enough to allow you to include other vendors if you choose. Currently, TEC maintains information on over 820 enterprise application solutions.
Step 2: Create and Issue the Request for Information (RFI)
Your selection team must now create an RFI in a format that will be easily understood by the vendors. Traditionally, this is done using spreadsheets. You should also determine a time limit for vendors to respond.
The RFI may include thousands of criteria listed, and if it is not properly organized, it can lead to many questions from the vendors—and possibly faulty interpretations by the vendors if line items are not completely clear. If all the vendors have not responded by the deadline, a decision will have to be made with respect to extending the deadline (which is often the case). Once again, this can be a time-consuming step and may lead to the elimination of vendors that should actually be in the running.
However, vendors that are part of TEC's knowledge bases have already submitted their responses to the standard RFI functionality, and their answers can be imported into the new RFI before sending it to the vendors. Vendors need only to quickly verify existing responses and respond to any additional criteria supplied by the user organization. This accelerates the process of obtaining vendor responses, and will ultimately shorten the selection process. TEC's RFI format uses industry-standard definitions; provides areas for prioritization, vendor responses, and supporting document attachments; and is easily loaded into TEC's decision support tools for further processing.
At the end of this step, as many as 20 or 30 RFIs should be returned with vendor responses, ready to be compared and evaluated against each other.
(Click here for larger version)
Figure 1: Vendor responses to an RFI
Step 3: Evaluate Vendor Responses to Create Your Shortlist
Once you've received vendor responses to your RFI, your next task is to evaluate the solutions in order to generate a list of two or three vendors that best match the functionality you require. The traditional spreadsheet method becomes a daunting experience at this stage, since as many as 30 vendors may have responded to close to 4,000 criteria, and each one must be compared. Manipulating all of this data with any degree of accuracy is almost impossible in any reasonable time frame. However, a shortlist must be created, since it is not possible to negotiate any final conditions, contracts, etc., with more than a few vendors. In most selection projects, the maximum number of vendors in the final negotiation and selection stages is three.
Vendor RFI responses can be imported into TEC's decision support tools and evaluated against each other to create a shortlist of two or three vendors that best match the prioritized requirements. This evaluation can be done at any level within the RFI; priorities can be adjusted and what-if scenarios can be run and re-evaluated as many times as deemed necessary.
The end result is achieved quickly and accurately using TEC's weighted average composite index (WACI). This index measures the consistency with which product capabilities correspond to client requirements and priorities. It also overcomes the risk of using weighted averages alone, which can often mislead users into selecting products that do not closely match their requirements, since weighted averages can conceal product performance inequalities.
(Click here for larger version)
Figure 2: Weighted average versus weighted average composite index (WACI)
Phase 3: How to Negotiate and Make a Final Selection
The first two phases narrowed the field down to the final two or three vendors best suited to provide the functionality you require. Your selection team must now prepare its negotiating strategy, critical terms and conditions, and maximum budgets—as well as be ready to enter into negotiations with the chosen vendors. The following steps will describe the final stages of selecting the winning solution.
Step 1: Conduct Vendor Demonstrations for Each of the Short-listed Vendors
Invite vendors on site to give a live demonstration of their products and to prove how their solutions suit the needs of your organization. Your selection team should prepare the vendors in such a way that the processes and requirements deemed critical to the business are showcased during the demonstrations.
Your selection team must include the champions and subject matter experts (SMEs) in this exercise so that they can evaluate the functionality, ease-of-use, and coverage of critical processes as expressed previously. You should devise a system of rating these elements so that comparisons can be made between the remaining solutions. This is where the whole exercise can break down—how-to processes will have to be revised to align with functionality. Needless to say, these are extremely important decisions.
At this point, what-if scenarios have to be determined and discussed. Differences of opinion between members of the selection team, champions, and SMEs will require negotiation—and compromise—in order to resolve the differences.
Typical vendor demonstrations include a canned demo put together by the vendor. This allows the vendor to show its software in the best light possible, as opposed to how it truly enables your business requirements. However, TEC's decision support tools assemble specific demonstration scripts for the vendor to follow. These scripts cover the product functionality decision criteria that have the greatest influence on enabling your business requirements. Your selection team will use the demonstration scripts to qualitatively score the vendors on their performance. Results are then loaded into TEC's decision support tools and become essential decision criteria for final vendor negotiations. What-if scenarios can also be created quickly and presented through graphics and reports to help the team better qualify arguments and explore all possibilities.
Figure 3: Demonstration scorecard
Step 2: Gather Client References for Each of the Short-listed Vendors
In addition to the scripted demonstrations, client references from each vendor are an important part of the selection process. These references allow your decision makers to interact with other organizations similar to yours that have implemented the same products. You can create a set of standard questions to ask each client reference. Responses provide insight into a vendor's track record and ability to deliver the required results, and can be loaded into TEC's decision support tools to be prioritized for objective analysis.
Figure 4: Client references loaded into decision support tool
Step 3: Issue Requests for Quotations
Each of the short-listed vendors will receive an official request for quotation, which is to be submitted by a deadline determined by your selection team. This quotation details the cost associated with implementing the software, and will include any additional costs for licenses, training, maintenance, and other service and support. This is usually the most critical information for your eventual decision, and can be loaded into TEC's decision support tools to allow for price-per-functionality analysis.
Figure 5: Price for functionality analysis
Figure 6: Cost comparison
Step 4: Conduct Final Negotiations
At this point, the selection team should have all the necessary information to make a recommendation to management. When approaching management, it is crucial to present a solid business case for the chosen solution. This involves measuring business value and risk, and illustrates due diligence that the best choice has been made by presenting a clear audit trail showing how vendors matched the established criteria at each stage of the process.
Your selection team should also enter into the final negotiations with the one vendor that it feels can best meet the needs of the organization. Often, price will be the key factor in the negotiation, but other aspects, such as implementation strategy and timing, ongoing support, and scalability, can be just as important. If not carefully considered, these factors can add to the cost of the solution and eat into any return on investment (ROI).
TEC's decision support tools provide you with custom, detailed final reports that audit every aspect of the selection. The tool helps provide analysis in combination with cost factors, and can create easy-to-understand summary results based on thousands of criteria. The final reports and analysis will provide the information necessary to support your selection team's recommendation, and arm you with the analytics that can be leveraged in the final negotiations.
(Click here for larger version)
Figure 7: Report samples
A Brief Note on Post-selection Activities
Post-selection delivery gap analysis is crucial, as it will indicate if there are any customizations or extensions that must be created to meet the unique needs of your organization's processes.
TEC's decision support system can be used throughout the implementation for such activities as tracking project progress, and verifying completeness in functionality, and can serve as a base for preparing test cases, prioritizing testing, and planning the implementation schedule. It also serves as a guide to set up user training on the new software system. All information resides in the same repository, and can be easily accessed to resolve questions that may arise during implementation.
This concludes the two-part series Your Guide to Enterprise Software Selection.
Are you looking to start a successful software selection project? Launch your project here.