Facing A Selection? Try A Knowledge-Based Matchmaker Part 2: A Timesaving Solution

Executive Summary

Selecting a piece of enterprise application software has never been an exact science. Vendors' hype, consultants' conflict of interest, users' doubts, tediously long selection processes, and unclear decisions rationale are some of the unfortunate watchwords for most selection practices.

It is daunting for corporate IT buyers to discern the true capabilities, strengths and weaknesses of a given enterprise application suite, given the propaganda that pervades vendors' endeavors to differentiate themselves (see Beware of Vendors Bearing Solutions). When making strategic IT acquisitions Buyer's project teams, inundated with an abundance of available products and technologies, have a difficult time translating the content of glitzy marketing slides and grandstanding presentations into the deliverable products.

Still, as a 'one-size-fits-all' product is still not quite a viable possibility, almost every product can win provided certain set of requirements. The Catch 22 for both buyers and vendors/VARs is to pinpoint the right opportunity in this ongoing 'dating game'.
In Part 1 of this article it was suggested that an effective RFI/RFP process can streamline the selection process avoid the pitfalls of past selection processes.
This is Part 2 of a four-part tutorial that addresses this situation from the viewpoint of the Buyers, Vendors, and Resellers.
Part 1 - Overview of the Problem
Part 2 - Presents a Solution
Part 3 - Presents examples of applying the Solution
Part 4 - Makes User Recommendations

From Words To Action

Given that a figure is worth thousands of words and that numbers are more irrefutable than postulates, it is a time to illustrate the discussion in Part 1 with a concrete example. Depending of which hat you wish to wear (or are already wearing), you may put yourself in a perspective of a prospective customer, vendor or consultant.

Epicor, QAD and Ramco Systems are three Tier 2, mid-market ERP vendors that are part of TEC's Discrete Manufacturing ERP Knowledge Base chosen for this particular exercise. TEC has compared the features and functionality of each of these vendors using WebTESS®, our patented decision support tool, which uses the Multi-Attribute Utility theory (MAU), Analytic Hierarchy Process (AHP) and TEC's patented decision science to compare vendors and products relative to one another in a statistically valid model (see our Selection Assistance section for more information). The tool will perform the role of the machine in the above-mentioned human-machine combination so that we can find out how these vendors compare in Financials, Manufacturing, Inventory Management, Human Resources, Purchasing, Quality, Sales, and Product Technology areas.

TheTEC Solution

TEC recently released its Discrete Manufacturing ERP Knowledge Base (KB), which includes 21 vendors rated on approximately 3,500 functional and technical criteria. The criteria have been isolated as meaningful to best differentiate ERP packages, based on TEC's past selection experiences. As the functionality scope covers many modes of discrete manufacturing (make-to-order (MTO), engineer-to-order (ETO), make-to-stock (MTS), lean manufacturing, etc.) and it goes beyond core ERP functionality as well (into e.g., sales force automation (SFA), product lifecycle management (PLM), etc.), and as the technological questions attempt to cover many technical aspects (e.g., general architecture, degree of integration among modules, interconnectivity, data protection and restoration, security features, tools, etc.), we believe that the number of criteria serves the purpose. A lesser number would likely fail to provide an accurate picture, while a greater number would involve mundane details (e.g., the maximum length of the 'item description' field, or the capability of the system to print on 8" x 11" paper size).

The KB is powered by WebTESS®.

Among the vendors that currently reside in the knowledge base are Epicor (NASDAQ: EPIC), QAD (NASDAQ: QADI) and Ramco Systems Limited. These vendors have submitted a Request For Information (RFI) document to TEC either through a particular selection project or voluntarily. The number of participating vendors is likely to increase in the near future, as many vendors are still in the process of submitting their RFI documents. If you believe that your product deserves exposure through the TEC RFI Center, please do not hesitate to contact TEC about your possible participation.

Figure 1 below lists the vendors and their corresponding application suites that are chosen for this example.

Figure 1:

Vendor     Application Suite   
Epicor  Vantage
Ramco  e.Applications


TEC analysts use WebTESS® to rank each vendor's ability to meet the functional and technical requirements outlined in the RFI. The first step in doing this is to organize the 3,500 RFI items into a hierarchical tree known as a Decision Hierarchy. The Figure 2 below illustrates the high-level criteria in the Decision Hierarchy for the Discrete Manufacturing ERP Knowledge Base.

Figure 2:

Within each of these high level groups (i.e. Financials, Human Resources, etc.) are subgroups of smaller sets of functional and technical criteria. For example, Figure 3 below indicates the subcategories under Financials.

Figure 3:

The next step in ranking the vendors is to prioritize the Decision Hierarchy. This involves indicating the importance of each criterion to a technology selection decision. The percentage in parentheses to the right of the criteria indicates the priority, or how much of the total decision is allotted to the criteria. For example, 12.50% appears next to Financials. This indicates that the sum of the criteria under Financials is responsible for 12.50% of the total decision. In Figure 3 above, all high level criteria are prioritized equally thus each high level criterion receives 12.50% of the total decision.

Each vendor in the Knowledge Base has a set of ratings (or scores) that correspond to the end level criteria in the Decision Hierarchy. The possible ratings appear in Figure 4 below.

Figure 4:

Responses Explanation
Supported Supported out of the box
Modification Requires modification
(non source code change)
3rd Party Support     Supported via a 3rd Party solution  
Customization Requires Customization
(source code change)
Future Release Supported in a Future Release
Not Supported Not supported whatsoever

WebTESS® converts the ratings into numerical equivalents and uses the equivalents in conjunction with the priorities to determine a weighted average. The calculation is as follows:

Weighted Average = Sum of ([Priority] * [Rating])
for the entire set of end level criteria.

For the baseline scenario all priorities are set equal, thus the priorities amongst siblings are the same at every level in the Decision Hierarchy. The Figure 5 below illustrates the priorities in the baseline scenario.

Figure 5:


The baseline rank and weighted average of the vendors given these priorities is as follows:

Figure 6:

Rank Vendor Weighted average
1 Ramco Systems   81.37
2 Epicor 75.99
3 QAD  74.69

These results indicate that when all priorities are set equal, Ramco Systems has the highest weighted average. Ramco Systems' score is 5.38 points higher than Epicor and 6.68 points higher than QAD. The results are not surprising given Ramco's broad horizontal functionality, as opposed to Epicor's and QAD's more pronounced vertical focus. These gaps are nonetheless relatively small, and a more detailed look at the results will reveal the strengths and weaknesses of each vendor.

The graph below indicates the contribution to the weighted average for each vendor across the high level categories in the Knowledge Base, given the baseline priorities. The scale is weighted average. The sum of each node equals the weighted average in Figure 6 (above) for each vendor.

The Contribution Analysis graph (Figure 7 below) indicates that although Ramco Systems is the overall leader, the vendor falls in second or third place in a majority of the categories. This is because QAD is the leader for such categories as Product Technology, Sales, Quality, Purchasing, Inventory and Manufacturing Management, but QAD does not support any Human Resources functionality. This causes QAD's overall score to fall below Ramco Systems and Epicor. Furthermore Ramco Systems has a significant gap in Product Technology compared to the other two vendors, given its focus only on Microsoft technology. Epicor is the clear leader in Human Resources and performs well in every category except Sales and Quality, where this vendor is clearly in third place.

Figure 7:

Determining An Appropriate Client

The results in Figures 6 and 7 were generated using the baseline priorities. From these strengths and weaknesses it is possible to build a profile of an appropriate client for each of these vendors. The profile consists of a set of priorities that match the vendors' strengths and weaknesses. Note that WebTESS® users can build unique profiles of their own organization to determine how any of the 21 vendors in the Knowledge Base rank, given their profile (business requirements and the accompanying priorities).

Accessing TEC's ERP Knowledge Base

The list of vendors currently present in TEC's Discrete Manufacturing ERP Knowledge Base (see it in action) includes:

Vendor Application Suite
Adonix X3
Baan iBaan
Epicor Vantage
Frontstep SyteLine
Glovia Glovia.com
IFS IFS Applications
InterBiz PRMS
J.D. Edwards OneWorld
Lilly Software Associates VISUAL
Made2Manage M2M Enterprise Business System
Microsoft Great Plains eEnterprise
Microsoft Great Plains Solomon IV
Navision Attain
Navision Axapta
(Formerly Damgaard Axapta)
Oracle Oracle Applications
Ramco e.Applications
Relevant Business Systems INFIMACS II
ROI Systems MANAGE 2000
SAP mySAP.com
Syspro IMPACT Encore

To accommodate different needs and/or budgets, TEC offers different options for accessing the TEC KB's content at http://webtess2.technologyevaluation.com :

  • To access one selected vendor's data for a week: $250
  • To access three selected vendors' data for a week: $600
  • To access the entire KB's content for a month: $5,000

This concludes Part 2 of a four-part tutorial on how to effectively streamline the ERP selection process.
Part 3 illustrates how, by changing the selection criteria, the vendor's scores will change.

comments powered by Disqus