Evaluation Center FAQs

A TEC Evaluation Center is an online environment for evaluating and selecting enterprise software solutions. Each Evaluation Center contains detailed information about a specific type of solution, such as enterprise resource planning (ERP), business intelligence (BI), etc. This information includes up to several thousand enterprise software features and functions—modeled to reflect real-world use—and ratings that indicate vendor support for those features and functions.

You can define your company's requirements by setting priorities for the features and functions modeled in a given Evaluation Center, and compare them to vendor capabilities to find out which vendor solutions best meet your needs. You can analyze solutions at any level of the feature/function hierarchy, display the results in easy-to-understand graphs and charts, and create detailed reports.

For more information about TEC's Evaluation Centers.

TEC's Evaluation Centers are much more than a simple directory of vendor features and functions.

To begin with, we model the features and functions in our Evaluation Centers to broadly reflect the way vendors organize them in their software offerings. We then include request for information (RFI) responses from vendors that tell us the extent to which their solutions support those features and functions. Vendors complete TEC's RFIs without a specific project in mind, so their responses more accurately reflect their product's capabilities.

Finally, we couple our extensive vendor data with our powerful decision support engine, ebestmatch™, and a user-friendly online interface so you can analyze, in great detail, how well different vendors meet your business requirements. All you have to do is input and prioritize those requirements.

A knowledge tree is the hierarchical structure of enterprise software criteria (features and functions), organized into logical categories that can reflect real-world business use. In the Evaluation Center, the criteria are presented according to that hierarchy so you can examine them at multiple levels of detail.

ebestmatch is the decision support engine behind TEC's Evaluation Centers. Any time you compare solutions in an Evaluation Center, ebestmatch performs the underlying computation and analysis using TEC's patented algorithms. It's the same underlying technology found in our desktop decision support software (DSS), ERGO.

A weighted average (WA) is the traditional way to measure the performance of a given solution. To calculate a solution's weighted average, you multiply its rating for each criterion (how well it performs) by the criterion's “weight” (the priority you've assigned to the criterion), and sum the results.

The problem with weighted averages is that high ratings on key criteria tend to compensate for low ratings on others—giving you a false sense of how well the solution meets your requirements.

Let's say you've prioritized three criteria as follows:

Maintenance: 50%
Functionality: 25%
Ease of use: 25%

And two vendors have ranked their solutions' support for these criteria like this:

The weighted averages would be as follows:

This makes it look as though both solutions support your requirements equally well, even though they don't.

To compensate for this imbalance, TEC uses the Weighted Average Composite Index (WACI). WACI calculations consider not only each solution's weighted average, but the consistency of its scores across all of the criteria involved in the decision. The WACI is expressed as a number between 0 and 1. The higher a solution's WACI, the more closely the solution's rankings match your priorities.

In this example, solution A might receive a WACI of 0.5, while solution B might receive a WACI of 1.

Multiplying a solution's WACI by its weighted average gives you the BestMatch Factor (BMF). Essentially, the BMF allows you to include the degree to which a solution matches your priorities in a weighted average calculation. Vendors that receive the highest BMF are the most appropriate vendors for your organization.

So in this case:

Now the difference is clear. Because it matches your priorities more closely, solution B is the better choice.

Our analysts set the default priorities based on general business scenarios.

Vendors must complete TEC's detailed requests for information (RFIs), containing up to several thousand enterprise software features and functions and described using industry-standard terminology. When a vendor completes an RFI, TEC's analysts compare the responses to industry benchmarks, contacting the vendor to discuss specific features and functions that need clarification. Once the RFI is approved, the vendor information is transferred to the appropriate Evaluation Centers.

TEC Certified Products have undergone additional scrutiny, beyond the standard request for information (RFI) response validation, and met our analysts' stringent requirements. To have a product certified, the vendor must deliver a formal product demonstration based on a script developed by our analysts. Based on the results of the demonstration, our analysts may adjust the product's RFI ratings to ensure that the product is represented as accurately as possible for use in our Evaluation Centers.

Learn more information about TEC certification.

TEC is not a referral service. We have no vested interest in helping you choose any one solution unless it's the best fit for your organization. To that end, we focus on the quality of the information we collect, and on ensuring that you and the vendors have a common understanding of the features and functions in our Evaluation Centers. Our analysts use industry-standard benchmarks to review all requests for information (RFIs) when conducting their vendor reviews. They regularly interview vendors about their products, and ask for demonstrations of features and function that fall outside the expected range of ratings.

Let us know what vendor and product you are looking for and we will work to get the information in our Evaluation Centers as soon as possible. If you need the information right away for a specific project, we can send out a customized request for information (RFI) on your behalf.

Contact our Selection Services team for details.

Anyone involved in the software evaluation and selection process can benefit from TEC's Evaluation Centers:

• Potential buyers can quickly determine their business requirements, conduct preliminary selections to identify a working list of vendors and products, and compare the weaknesses in their existing systems to the strengths of a new system—all outside the sphere of influence that may be present in a vendor's marketing collateral or a consultant's alliances.
• Existing users can evaluate their current systems, and justify recent selection decisions with detailed reports.
• Consultants can help their clients select enterprise software solutions using the analysis tools, data, and reports in the Evaluation Centers.
• Vendors and their system integrator partners can justify whether to compete in a particular selection project, and conduct competitive analyses.
• Analysts, educators, and students can leverage the information in TEC's Evaluation Centers to support analyses or create detailed reports.

The free trial gives you access to TEC's Evaluation Centers for a limited time. During the trial period, you can use nearly all of the Evaluation Center functionality to compare several vendors and see how well their products meet your requirements.

However, given the number of variables and criteria that you must consider to accurately evaluate enterprise software systems, we recommend that you purchase extended time as part of a complete evaluation and selection project. A thorough evaluation will help you ensure that your shortlist includes only vendors that can deliver solutions that meet your requirements.

Contact our Selection Services team for details.

Because enterprise software system is a such huge investment, you should never make a purchase decision without first meeting face-to-face with shortlisted vendors, discussing specific capabilities, talking to references, visiting live site implementations, and conducting scripted demonstrations.

TEC's Evaluation Centers can help you incorporate the information you collect during these meetings, visits, and demos into your decision. You can analyze it objectively, along with the vendors' support for your requirements.