If you're in the early phases or even in the midst of a software selection project, I assume you've read industry analyst prestidigitations, squared those up with vendor posturing, and lent an ear to various experts.
Those efforts might help you figure out whether the products you're considering buying and eventually implementing actually do what you need, but the new TEC Focus Indicator (TEC FI) is a concrete way to start gauging the real functionality, competitive differentiators, and focus of enterprise software products.
The TEC FI shows you how a product differentiates itself (in terms of functionality) from other products on the market. It will give you an indication of whether a vendor has focused its product toward the sort of functional needs that your company has.
Over the last year, we've been publishing product certification reports, which now feature the new TEC FI graph. These reports provide background on each vendor's business, our analysts' experience witnessing how the product works, and some insight on the product's functionality. It's that particular insight on the product functionality I'd like to explain through this discussion about the TEC FI. You may have seen some early versions of this graph, but it's evolved. I'll explain what the TEC FI signifies and how you can use it to supplement your research with information not available elsewhere.
Why TEC Made the FI
When planning to buy some type of enterprise software system, have you ever been told by a vendor that there's little sense comparing products from a functional perspective? That in fact, what matters is some other issue, like the ongoing support, or ultra-friendly user interface of their own product—because really, any vendor can offer you the functionality you need? In other words, the vendor is claiming that the market for its product is commoditized and that the real value comes from something other than features and functions.
It's possible that vendors making that claim are correct. Factors like the user interface, professional services, support, vendor references, implementation methodology, not to mention cost, are necessary to consider before deciding on a particular software solution. By all means, include these in your selection decision.
Nevertheless, I disagree that comparing functionality isn't worthwhile. Even in mature software markets like the enterprise resource planning (ERP) space, products differ greatly in terms of the functionality they provide. I will prove this shortly.
The essential question for a company trying to select software remains: will it actually help us do the things we need to do? And that question is of course answered by the functionality provided in the software product. There's no escaping that at some point you must compare the functionality between the products you're considering, and make sure they provide what your company needs.
Even if you already have software in place and are not necessarily thinking of purchasing new software, companies often keep abreast of the software market in order to consider what their software currently provides in relation to new innovations and evolving technologies.
Although the TEC FI is not designed as an in-depth evaluation tool, it will help you down that path because it directly corresponds to all the deep functional criteria TEC provides for evaluating software.
Reading the TEC FI: Some Examples
We create a TEC FI for each product that we certify. The TEC FI measures a product's overall level of support for functionality (as modeled in TEC's research) against TEC's benchmarks of what you should expect to find across the market space as a whole.
We identify the benchmarks as colored zones on the graph. The lowest level of support would be completely unsupported (with a value of 0). The greater the level of support for the defined software functionality a vendor provides, the more points its product earns.
As a product provides increasing support for a particular category of functionality, it climbs the graph toward the Competitive Zone. Finally, if it provides a greater level of functional support than most other products on the market, it moves into the Dominant Zone. We call this its functional focus—in other words, the more a product supports functionality in a particular category, the more it's focused on that category.
Figure 1. Functional Focus in the TEC FI
Let's look at a few TEC FI graph examples from our research on real ERP systems for companies in discrete manufacturing industries. In order not to distract from the explanation, I've anonymized the product names as Product A, Product B, and Product C.
Product A
In the first example, we're examining Product A, which has been on the market for a number of years. The red dots represent the level of support provided in the product for seven categories of functionality in our model.
Figure 2. TEC FI for ERP Product A
It's clear that in most categories, the product is within the Competitive Zone. This means that this product's functionality is within the range of what you should expect to find from other products. Note the Industry Average circle, which is a normalized average of all products' levels of support. In some cases, Product A is slightly above or below the industry average, which shows you where Product A offers a bit more or less than what you'd normally expect to find on the market.
Perhaps the more interesting thing about this product is its quality management and human resources (HR) scores. Both these modules are in the Dominant Zone. This means that the product offers a significantly higher level of support for these types of functionality than competing vendors. Not only that, but the product actually provides the maximal possible level of support for quality management functionality.
The vendor delivering this product must have reason to put so much emphasis on developing quality management and HR. Perhaps these features were developed in response to the vendor's user base, or else the vendor is targeting new business in industries with heavy requirements in quality management or HR. Either way, this is useful information for your search.
If your company needs common ERP features to support its business but has a particular requirement for quality management or HR, this product would be a good contender to delve into more deeply. Compared to other products in the market, the breadth of functionality that stands out the most, and most differentiates it from its direct competitors, is the HR and quality management functionality. The significant way in which these stand out versus the average product on the market proves that if you were to compare this product to others, you will see vast differences in functionality.
Product B
The following TEC FI represents an ERP solution from a vendor that is a more recent entrant to the market. Like Product A, it targets the manufacturing industry, but its FI is quite different from Product A's.
Figure 3. TEC FI for ERP Product B
In this case, the most obvious difference between the products is the HR module. For Product B, the HR module sits within the Minimal Support Zone. This vendor provides very little functionality to satisfy HR requirements.
This may be because it's a newer vendor and hasn't developed the functionality yet, or it may be a strategic decision to focus on other ERP modules, assuming clients will use separate software solutions for their HR needs. However, even if your company already has an HR system that you’re happy with, you'd still need to devote time later in the evaluation process to figure out how the vendor would connect its product with your HR system. On the other hand, if you need your ERP system to include a full-fledged set of HR features, the fact that this product does not provide them might be a show-stopper for you.
Comparing Product B to Product A shows that the two approach support for quality management and HR from different perspectives, but are competitive for the other areas. There are nevertheless, some small differences in those areas. For example, Product B’s inventory management is above the industry average but Product A’s is below the industry average. However, if you care about neither HR nor quality management, then either of these products could be a fit for your needs.
In a real-world situation, where you actually compare these two products in-depth, these TEC FI graphs ought to prompt you to ask the vendors some questions about their product roadmaps and development strategies. For example, if one vendor has been on the market longer than the other, how come both hover in the Competitive Zone for the same functional modules? Product A has been around longer but doesn't seem to be developing its inventory management or manufacturing management functionality as deeply as it has its HR and quality management. Does that mean the vendor is steering the product away from environments with more intensive manufacturing requirements? Checking the vendor's references might reveal more about how the product has developed to address modern manufacturing problems.
On the other hand, based on the scores of Product B, which comes from a newer vendor in the market, you might ask about the rapidity with which it was able to have its product compete so squarely with others. Is the product growing along a trajectory that fits your organization? Can the vendor offer appropriate support for so much functionality? How about the lack of HR and quality management? Will the currently low levels of support for these modules affect your own company's growth? These questions aside, Product B appears to offer a greater quantity of support for managing manufacturing operations, which reveals the sort of company this vendor is targeting.
In case you still think that these graphs, with many modules placed in the Competitive Zone, tend to support claims that the ERP market really is commoditized and there aren't many significant functional differences between products, look at this third TEC FI, for what most people consider a tier-one product.
Product C
Product C is by all definitions, a mature solution that offers just about everything imaginable from a functional perspective.
Figure 4. TEC FI for ERP Product C
The red dots cluster toward the center of the graph, meaning that each module dominates when comparing the product to most others in the market. This shows that there is still quite a bit of space for the other products to grow. And in fact, only two modules—financials and purchasing management—provide the maximal level of support, so even this dominant product has space to develop more functionality, which other products already provide.
This type of FI might also be a concern for some companies. The depth of support for functionality provided by this product comes at a premium cost.
I'd like to explain three things to add clarity to these graphs and give some context to the information they display: 1) how TEC models software functionality, 2) how TEC creates the benchmarks for a given market, and 3) what TEC certification is.
1) How TEC Models Software Functionality
When I say that we model different types of enterprise software, what I mean is that we research and identify as many of the features and functions provided by a type of software system as possible, and then organize them into logical groupings of functionality.
For example, in our model of an ERP system, the manufacturing component has categories called Product Costing, Shop Floor Control, Production Planning, etc. Each of these categories has subcategories that group criteria representing the individual features provided by the software. How many criteria are there? As of this writing, our mixed-mode manufacturing ERP model covers 3,739 criteria. On the other hand, our product lifecycle management (PLM) model includes 1,768 criteria and our supply chain management (SCM) model includes 2,210 criteria. The number of criteria varies depending on the type of software, since different systems require different quantities of criteria to model correctly.
The point is that we're modeling the general structure of a given type of software so that a potential buyer of that software can review what it provides and compare it against competing products on a fair and equivalent basis.
We use two main sources for determining the criteria and structures of our models. One source is software vendors. Through product demonstrations, inquiries, and ongoing conversations with software vendors, we compile a significant amount of data on the various forms of functionality developed for a particular market. In addition, our work assisting companies through their software selection projects provides an important real-world source of information on the types of functionality companies use and need.
2) How TEC Creates Its Benchmarks
We maintain data on how vendors in each market space (ERP, SCM, business intelligence [BI], customer relationship management [CRM], PLM, HR, etc.) provide support for all the features we've modeled (i.e., those thousands of criteria that I referred to previously). Using this data, we're able to determine which functionality is usually supported by vendors and to what degree they support it. For example, does the vendor support a given feature outright, or do they need to make some modifications to support it? We use eight different ratings (with accompanying numeric values) to determine the level of support for each criterion.
To give some context to the source of these benchmarks, in the ERP space for discrete manufacturers we've modeled over 3,000 data points on the functionality. At the time of this article’s publication, we've gathered data from roughly 95 vendors that support those criteria to varying degrees, and of those, we've certified 54 (more on certification later). We include all types of vendors, from large tier-one companies to niche vendors.
Knowing the most commonly supported functionality is significant because that's what allows us to create benchmarks to measure the products against. We start by calculating the industry average, or the level of support that is most commonly provided across all software products competing in the market.
Next, we define some zones as demarcation points for determining where a product offers a lower quantity of functionality than you'd expect from the market, about the same as what you'd expect from the market, and where a product offers a significantly higher quantity of functionality than you'd expect from the market. As noted previously, we call these the Minimal Support, Competitive, and Dominant Zones.
We define the zones using a bell curve. The Competitive Zone comprises the middle portion where the most common level of support occurs across the market; of course the Industry Average line is normalized at the apex. The Minimal Support and Dominant Zones are at each of the low ends of the curve.
We consider that any product providing a level of support below the Minimal Support line only offers the bare essentials, or else is simply not competing for customers that need that sort of functionality. Products that offer more than the minimally supported range of functionality are competitive within the market.
The Dominant Zone is the demarcation point for products that provide what we consider to be competitive differentiators. Namely, this product provides a level of support for functionality that is uncharacteristically greater than what you can generally find in the market.
Using these benchmarks is valuable toward forming a set of baseline expectations for what you want from a product. If your company is looking for an ERP system to support some intensive financial management requirements as well as basic HR and sales management needs, but you have no manufacturing operations, then you're more likely to find what you need from a system that scores high in either the Competitive Zone or the Dominant Zone for financials and remains at least competitive for HR and sales management (it is obviously not a deal-breaker if it provides minimal to no support for manufacturing management).
3) TEC Certification
I've explained our research models and how we create the benchmarks for measuring products in a market, but I haven't discussed our certification process. TEC Certification is important because we only produce TEC FIs for certified products.
TEC Certification is a process whereby we review a vendor's product, with the vendor’s collaboration, to ensure that our data represents the product as accurately as possible. Because we've actually seen these products demonstrated and validated a representative sample of their functionality, we have a high level of assurance that our benchmarks are correct.
If you've ever issued a request for information (RFI) to a vendor, you know it can be a complex and difficult document to respond to. It's hard not to feel pity for the people responding to RFIs. They have to interpret questions from people trying to express what they want from a complex software system. Even industry-standard terminology can sometimes be expressed in confusing contexts.
Thus, anybody who has been involved in an RFI process knows there is a chance vendors might misinterpret the questions. Misinterpreting a question means a vendor might provide the wrong answer about what its product does. This is the sort of pitfall TEC Certification is designed to minimize.
We base our product certifications on questions from our standard, public RFIs. Our RFIs are identical to our models of software functionality. Essentially we ask vendors to demonstrate how their products fulfill a sampling of functionality. We choose the representative sample from all the categories in our model. When the vendor demonstrates its product to our analysts, this provides the opportunity for our analysts to see that the vendor has truly understood the criteria and responded accordingly. In cases where the vendor has misunderstood, the analyst corrects the response (to either a lower or higher rating, depending on what the product actually does).
It's important to note that the certification process is a paid service we offer software vendors. In other words, a vendor pays TEC to ensure its product is being represented as accurately as possible to the audience of people evaluating it through our public evaluation system. Vendors have an incentive to pay for certification because when companies like yours take advantage of TEC's RFI in a software selection process, the vendor doesn't have to expend the huge amount of effort interpreting and trying to respond accurately to your RFI. They already know they've understood what you're asking and we've helped them answer you appropriately. Vendors decrease their risk of losing business due to delayed or incorrect RFI responses. Furthermore, ensuring that the vendor and client are on the same page at the RFI stage will save time when preparing for implementation.
Revealing the Secret Cabal behind the TEC FI
This article would be more intriguing if there were a cabal. There isn’t—TEC is boring in this way. Still, I expect that you're familiar with the criticism people sometimes lobby against analyst firms, suggesting that the firms' findings are influenced by vendors paying for their rankings, or that the firms cannot back up their claims with factual data.
It's not just potential buyers that criticize analysts this way. Software vendors do as well. It takes little effort to find recent examples of software vendors accusing or even suing an analyst firm with respect to some misgivings about the research results the analyst firm published.
To counter those types of criticisms, we make public the data used to generate TEC FIs. We are transparent and encourage you to review the data for yourself in our free, public Evaluation Centers. For every TEC-certified product for which you see a TEC FI, you can also review and evaluate the details of its functionality. Nothing is hidden, and the data is available for scrutiny.
The TEC FI is unique. High-level information that you glean from the TEC FI is based on thousands of functional criteria about products from all areas of the market. Information from other comparison graphics usually address a select range of vendors with respect to a few hundred criteria on things like a vendor's sales strategy, services, market presence, ability to execute, and some functional factors. These other types of graphics have a use, but they aren't designed to address product functionality in a deeply meaningful way. Rather, they're designed more as snapshots of a market, or as lightweight product evaluation tools.
They can help you identify vendors and products within the market but you'll require supplemental information if you wish to assess how well a product meets your company's needs. The TEC FI is a simple graph revealing the functional competitive differentiators of a particular enterprise software product against what is available on the market.