TEC's Evaluation Centers are much more than a simple directory of vendor features and functions.
To begin with, we model the features and functions in our Evaluation Centers to broadly reflect the way vendors organize them in their software offerings. We then include request for information (RFI) responses from vendors that tell us the extent to which their solutions support those features and functions. Vendors complete TEC's RFIs without a specific project in mind, so their responses more accurately reflect their product's capabilities.
Finally, we couple our extensive vendor data with our powerful decision support engine, ebestmatch™, and a user-friendly online interface so you can analyze, in great detail, how well different vendors meet your business requirements. All you have to do is input and prioritize those requirements.
A knowledge tree is the hierarchical structure of enterprise software criteria (features and functions), organized into logical categories that can reflect real-world business use. In the Evaluation Center, the criteria are presented according to that hierarchy so you can examine them at multiple levels of detail.
ebestmatch is the decision support engine behind TEC's Evaluation Centers. Any time you compare solutions in an Evaluation Center, ebestmatch performs the underlying computation and analysis using TEC's patented algorithms. It's the same underlying technology found in our desktop decision support software (DSS), ERGO.
A weighted average (WA) is the traditional way to measure the performance of a given solution. To calculate a solution's weighted average, you multiply its rating for each criterion (how well it performs) by the criterion's "weight" (the priority you've assigned to the criterion), and sum the results.
The problem with weighted averages is that high ratings on key criteria tend to compensate for low ratings on others—giving you a false sense of how well the solution meets your requirements.
Let's say you've prioritized three criteria as follows:
Maintenance: 50%
Functionality: 25%
Ease-of-use: 25%
And two vendors have ranked their solutions' support for these criteria like this:
Solution A
Maintenance: 50
Functionality: 100
Ease of use: 100
Solution B
Maintenance: 100
Functionality: 50
Ease of use: 50
The weighted averages would be as follows:
Solution A Weighted Average Score
50% (50) + 25% (100) + 25% (100) = 75
Solution B Weighted Average Score
50% (100) + 25% (50) + 25% (50) = 75
This makes it look as though both solutions support your requirements equally well, even though they don't.
To compensate for this imbalance, TEC uses the Weighted Average Composite Index (WACI). WACI calculations consider not only each solution's weighted average, but the consistency of its scores across all of the criteria involved in the decision. The WACI is expressed as a number between 0 and 1. The higher a solution's WACI, the more closely the solution's rankings match your priorities.
In this example, solution A might receive a WACI of 0.5, while solution B might receive a WACI of 1.
Multiplying a solution's WACI by its weighted average gives you the BestMatch Factor (BMF). Essentially, the BMF allows you to include the degree to which a solution matches your priorities in a weighted average calculation. Vendors that receive the highest BMF are the most appropriate vendors for your organization.
So in this case:
Solution A's BMF
WACI of 0.5 (WA of 75) = BMF of 30
Solution B's BMF
WACI of 1 (WA of 75) = BMF of 75
Now the difference is clear. Because it matches your priorities more closely, solution B is the better choice.
Our analysts set the default priorities based on general business scenarios.
Vendors must complete TEC's detailed requests for information (RFIs), containing up to several thousand enterprise software features and functions and described using industry-standard terminology. When a vendor completes an RFI, TEC's analysts compare the responses to industry benchmarks, contacting the vendor to discuss specific features and functions that need clarification. Once the RFI is approved, the vendor information is transferred to the appropriate Evaluation Centers.
TEC is not a referral service. We have no vested interest in helping you choose any one solution unless it's the best fit for your organization. To that end, we focus on the quality of the information we collect, and on ensuring that you and the vendors have a common understanding of the features and functions in our Evaluation Centers. Our analysts use industry-standard benchmarks to review all requests for information (RFIs) when conducting their vendor reviews. They regularly interview vendors about their products, and ask for demonstrations of features and function that fall outside the expected range of ratings.
Because enterprise software system is a such huge investment, you should never make a purchase decision without first meeting face-to-face with shortlisted vendors, discussing specific capabilities, talking to references, visiting live site implementations, and conducting scripted demonstrations.
TEC's Evaluation Centers can help you incorporate the information you collect during these meetings, visits, and demos into your decision. You can analyze it objectively, along with the vendors' support for your requirements.