Evaluation Centers and Knowledge Bases

Q. What is the difference between an evaluation center and a knowledge base?
Q. How are the evaluation centers different from other sites that list vendor data?
Q. What is a knowledge tree?
Q. What is eBestMatch?
Q. What is the difference between a weighted average and the Weighted Composite Index (WACI)?
Q. What is the BestMatch Factor?(BMF)?
Q. How are the default priorities in the knowledge base established?
Q. How do vendors qualify for an evaluation center?
Q. What is a certified vendor?
Q. How do I know the information in your knowledge bases is impartial?
Q. I'm looking for data on a certain vendor, but it is not in your knowledge base. How can I get this information?
Q. Who benefits from using TEC's evaluation centers?
Q. What type of information will the free trial provide? Is this enough information to create a shortlist?
Q. Should I base my purchase decision only on the information found in the evaluation center?
Q. I'm planning an evaluation, but I'm not sure if I should purchase a custom (online) evaluation or ERGO.

Application

Other questions? Contact Us

Evaluation Centers and Knowledge Bases

Q. What is the difference between an evaluation center and a knowledge base?

A: An evaluation center houses TEC's knowledge bases, which contain thousands and thousands of vetted and ranked criteria on different enterprise areas. When users enter any of TEC's two dozen (and growing) evaluation centers, they are prompted by TEC's shortlist wizard for their enterprise software and service requirements.

A knowledge base models the characteristics of a particular research topic. It is a repository of information organized in a comprehensive, hierarchical tree of evaluation criteria. Storing data about vendors or providers in the knowledge base allows for the effective organization of business needs and lets users focus their priorities on different subjects of their choosing.

By evaluating the user's prioritized criteria with the criteria ratings housed in a knowledge base, a list of vendors best matching the user's needs may be generated. Additional tools and analyses, such as product differences, product comparison reports, what-if scenarios, and sensitivity analyses allow the user to determine which solutions require further consideration.

back to top

Q. How are the evaluation centers different from other sites that list vendor data?

A: Unlike other sites, the evaluation centers are more than a directory of vendor features and functionality. Instead, TEC's evaluation centers provides extensive vendor data coupled with the powerful analytical capabilities of TEC's decision support system. Vendors complete TEC's requests for information (RFIs), which contain thousands and thousands of criteria, without a specific product in mind, so their responses more accurately reflect their product's capabilities. The vendor data is vetted by our analysts against industry benchmarks to ensure accuracy and impartiality. Suspect responses are asked to be demonstrated.

For more information about TEC's decision support system and its process methodology, see TEC's Software Selection Process.

back to top

Q. What is a knowledge tree?

A: A knowledge tree is the hierarchical structure of criteria, organized into logical categories. It is the navigation structure of the knowledge base.

back to top

Q. What is eBestMatch?

A: eBestMatch is the engine which employs TEC's patented algorithms used in both the online evaluation centers and the ERGO desktop decision support system.

For more information on TEC's decision support systems

back to top

Q. What is the difference between a weighted average and the Weighted Composite Index (WACI)?

A: The weighted average is a measure of the performance of a solution. It is calculated as the sum of the per criteria performance of a solution, multiplied by the numerical equivalent of your priorities. While it is a useful number and easy to calculate, it does not compensate for any imbalances within the weight of the criteria.

Weighted Average Composite Index (WACI) calculations (in the online evaluation centers and ERGO) are most useful when differentiating two solutions with close weighted averages. These calculations take into account the consistency of scores across all of the criteria in addition to the decision alternatives' weighted averages. The algorithms of eBestMatch engine generate the WACI, which is a number between 0 and 1. A WACI of 1 indicates that an alternative's scores were very consistent (in other words each criterion received the same rating). A WACI below 1 indicates inconsistencies in the scores. The degree to which the scores are inconsistent determines the WACI.

For more information, see Ensuring Accuracy: Weighing Priorities

back to top

Q. What is the BestMatch Factor?(BMF)?

A: The BestMatch Factor™ (BMF) is a number used to indicate the appropriateness of a vendor or provider for a user's requirements. It is a calculation of the weighted average multiplied by the WACI. This number is used to measure the vendor's overall performance against the pattern match. Vendors that receive the highest BMF are the most appropriate vendors for the user.

back to top

Q. How are the default priorities in the knowledge base established?

A: The default priorities are set by our analysts, based on general business scenarios.

back to top

Q. How do vendors qualify for an evaluation center?

A: Vendors must complete TEC's detailed RFIs which contain thousands and thousands of criteria that use industry accepted terminology. Once completed, TEC's analysts will review the data against industry benchmarks, and will contact the vendor's to discuss features and functionality to ensure the information is accurate. Once approved, the vendor information is transferred to the appropriate knowledge base.

back to top

Q. What is a certified vendor?

A: TEC certified vendors have met the stringent requirements set forth by our analysts. Associate vendors have taken the first step toward their TEC certification by completing our extensive request for information (RFI). Knowledge base responses come from the vendor representative who is responsible for replying to client RFIs, so TEC's receives valid and pertinent information about what the vendor can provide. TEC's analysts then validate the information against known vendor benchmarks and analyst knowledge and experience. Suspect or unclear responses are discussed with the vendor and asked to be demonstrated.

back to top

Q. How do I know the information in your knowledge bases is impartial?

A: Because TEC is not a referral service, it has no vested interest in what solution or service is selected, so its focus is on the quality and accuracy of the information collected. TEC's analysts use industry standard benchmarks to review all RFIs when conducting their third party reviews. All vendors are contacted and interviewed about their product. If a feature or functionality falls outside the norm, TEC's analysts will contact the vendor for more information or ask for a demonstration. Additionally, because vendors are not responding to a specific project, their responses are more reflective of their capabilities.

back to top

Q. I'm looking for data on a certain vendor, but it is not in your knowledge base. How can I get this information?

A: Let us know what vendor you are looking for and we will work to get their information in our knowledge base as soon as possible. If you need the information right away for a specific project, we can conduct a customized assignment for your project. Contact us for details.

back to top

Q. Who benefits from using TEC's evaluation centers?

A: Potential buyers can rapidly determine their business requirements, conduct preliminary selections to shortlist vendors, and compare the weaknesses in existing systems with the strengths of a new system, all outside the sphere of influence that may be present in a vendor's marketing collateral or a consultant's alliances. Existing users can justify their selection decisions with tailored reports. Vendors and their system integrator partners can justify whether to compete in a particular selection or conduct competitive analyses. Analysts, educators, and students can also leverage the information in TEC's evaluation centers to aid in analyses or create detailed reports.

back to top

Q. What type of information will the free trial provide? Is this enough information to create a shortlist?

A: The free trial gives users access to TEC's decision support system for a limited time with several vendors of your choice. You can change priorities, drill down to any level of information, and conduct numerous analyses, including a sensitivity analysis, what-if scenarios, product differences, and product comparisons. However, to take full advantage of the sophisticated analysis tools, we recommend that users purchase extended time. Given the number of variables and criteria that must be considered when conducting a complete enterprise system evaluation, additional time using the decision support system will help generate an accurate assessment.

Purchase an evaluation license

back to top

Q. Should I base my purchase decision only on the information found in the evaluation center?

A: Because an enterprise system is a huge investment with enormous business potential and risk, users should meet face-to-face with shortlisted vendors, discuss specific capabilities, talk to references, visit live site implementations, and conduct live, scripted scenario demonstrations. TEC's evaluation centers are an important element in the decision and should be used to aid these processes.

For more information on the evaluation and selection process see Streamlining Value in TEC's process methodology.

For a list of TEC approved consultants contact us

back to top

Q. I'm planning an evaluation, but I'm not sure if I should purchase a custom (online) evaluation or ERGO.

A: It depends on your goals. If you would like to quickly get started with a wealth of vendor data and a predefined set of criteria (customizable for your requirements), and you would like the evaluation process to be easily accessible to many people in your project team, then an online evaluation may be just right. However, if you would like to write your own criteria and organize them into your own hierarchy from a powerful desktop application, ERGO would be the compelling choice. Either way, the data is interchangeable. A knowledge base that you develop in ERGO can be used in an online evaluation center. Likewise, your online selection project can also be loaded into ERGO. TEC professional services experts are available to help you make this choice and guide you through the process.

Purchase an evaluation license

back to top

Application

Q. What happens after my access period expires?

A: You can purchase more time and continue to work on your evaluation without fear of losing your work. You may choose from the existing evaluation center licenses offered, or contact us to create a package customizing the time and number of vendors you need to evaluate for your project.

back to top

Q. How do I fine-tune my priorities after I set them in the Set Priorities screen?

A: You can adjust the priorities at any level on the knowledge tree from the Prioritize Needs section, under the Additional Features section.

back to top

Q. I want to export data from the evaluation centers to use in my presentations. What do I do?

A: The evaluation centers engine allows for the creation of detailed PDF reports, including sensitivity analyses, what-if scenarios, product comparisons, and product differences reports. These and other reports can be printed from the evaluation center if a package has been purchased. Other reports are also available, such as the software evaluation reports, and custom vendor viability and market data reports.

Purchase an evaluation license

back to top

Q. How do I view the results of specific sections in the knowledge tree?

A: Users can navigate through the knowledge tree by simply clicking on criteria names in the knowledge tree. Navigation through the hierarchy is akin to navigating through a file manager hierarchy (as in Windows Explorer).

back to top