TEC Selection Uses Knowledge Bases
we beg to differ that certain selection service providers' (including TEC's)
methods are archaic but they rather save time and cost, while taking care of
the functions and features "inventory management." To that end, TEC released
a number of knowledge bases (KBs), accessible through its evaluation centers
(e.g., ERP Evaluation Center www.erpevaluation.com,
CRM Evaluation Center www.crmevaluation.com,
etc.), each of which includes several dozen vendors rated on thousands of functional
and technical criteria. The criteria have been isolated as meaningful to best
differentiate these enterprise packages, and are based on TEC's past selection
experiences. For example, as the discrete ERP functionality scope covers many
modes of discrete manufacturing (make-to-order [MTO], engineer-to-order [ETO],
make-to-stock [MTS], repetitive manufacturing, etc.) and it goes beyond core
ERP functionality as well (e.g., sales force automation [SFA], PLM, etc.). Because
the technological questions attempt to cover many technical aspects (e.g., general
architecture, degree of integration among modules, interconnectivity, data protection
and restoration, security features, tools, etc.), we believe that the number
of criteria serves its purpose. A lesser number would likely fail to provide
an accurate picture, while a greater number would involve mundane details (e.g.,
the maximum length of the item description field or the capability of the system
to print on an 8" x 11" paper size).
KBs are powered by the eBestMatch decision making tool (formerly
WebTESS). How does all of this work? In a gross simplification,
behind the scenes there are databases which maintain functions and features
by software component. Correspondingly, there are databases of software vendor
capabilities aligned along the same lines. After you have updated the functions
and features according to your needs, rankings, and priorities, the two sets
of databases are matched to find vendors who best meet your criteria. The algorithms
and modeling techniques used to complete this "best match" tend to be sophisticated
and rival those used in advanced planning and scheduling (APS). The vendors
have submitted responses to the respective request for information (RFI) documents
to TEC either through a particular selection project or voluntarily.
is Part Three of a three-part note.
One presented the overview.
Two compared business processes to the feature/function approach.
Using a Knowledge Based System
Experience has shown that close to 95 percent of functional and technical requirements show up time and time again, and TEC has attempted to capture these, which provides repeatability and saves everyones time. The remaining 5 percent or so are requirements peculiar to your business and industry, which may happen to be fatal flaws (e.g., Wal-Mart electronic data interchange [EDI] standards in retail, or multiple-children bills of material [BOM] in the electronics industry), which need to be defined from scratch and prioritized appropriately. But, these should only be a small fraction of the entire immense RFI effort. Likewise, vendors' and VARs' effort in filling the new RFIs should only be limited to filling that delta document. Why haven't these been captured beforehand? For two main reasons:
The expertise across over twenty industries is not likely to be found at any
selection service provider, despite extensive horizontal knowledge (with a
deeper knowledge of a couple of particular industries) of their analysts.
The fact that every vendor that is serious about its vertical focus has separate
dedicated industry product leaders, special interest groups (SIGs), and so
on, speaks volume about how involving the vertical industry product extensions'
delivery is; and
Many industry specific functions still cross several industries, for example,
lean management practices like kanbans have permeated almost every manufacturing
environment. Thus, even the vendors that tend to develop vertical extensions,
sometimes referred to as industry service packs, periodically review these
individual functions for common use, and then decide to roll them back into
the standard horizontal (foundation) solution that is applicable across the
This is a continuing, open-ended system where similar solution packs will gradually be added to the above knowledge bases. Vendors and VARs that consider themselves leaders in certain industries, are more than welcome to contribute their suggestions toward building these add-ons, at least for the reason of differentiating themselves from the better rounded "horizontal" competitors.
Once the best product fit is found, it is common that the selected software still cannot meet all the requirements of the business, which calls for a decision to either change the requirement so the package can accommodate the business, modify the software to match the business need, or to design the business processes to work-around or account for the difference. If significant functional gaps exist between the system and the needs of the business, has the right product been selected? Does the right standard product even exist? If these gaps fall within the list of fatal flaws, the system will never be successful.
in a hypothetical case of two vendors differing by only a few percentage points
of required functionality, it is very likely that each of these differences
will carry a significant weight and could signal a requirement for an extensive
modification effort and expense. The ramifications of this kind of selection
are well-known (see Should
You Modify an Application Product?). The cost of modifications will be rolled
into the total cost of the proposal and will offset what could be viewed as
a vendor's amenability to modifications. The proposal costs to include software,
annual maintenance, implementation assistance, and modifications can now be
evaluated against the functionality scores. If a vendor has a high functionality
score but is also high in terms of total costs, you must determine if the costs
justify the functionality.
Beware Software Bloat
On the other hand, functionally richer vendors should not relax and expect a hands-down victory, while less functionally robust vendors should not be dejected or feel inferior. Indeed, despite many vendors' and industry experts' contentions of an inevitable functional parity convergence between products, a one-size-fits-all product is still not quite a viable possibility, and thus, almost every product can still win provided certain sets of requirements. There is always a risk of "as little modification as possible due to a rich standard functions inventory" approach coming with the price of "software bloat" and inflexibility.
Standard application software, for the most part, has been designed to meet the needs of multiple companies in multiple industries. To meet the needs of different industries, vendors have added many industry specific functions. However, normally, no single user company operates in more than one or two different industries. Although the fit is improved in one industry, the cost of this fit is feature bloat, meaning that all other industries must suffer the consequences of these non-essential functions. Bloat, in the form of complex program logic is required not to execute the function but to determine which of many paths the program must take. The cost of bloat is real. Code complexity leads to quality problems. Code complexity means support becomes more difficult. Enhancements and modifications prove more costly and error prone. Implementations are more lengthy and difficult. Upgrades from release to release become more time consuming and error prone.
software needs to evolve so that it recognizes the reality that businesses are
unique, but not by adding software bloat to existing products. Vendors also
need to build lean products that serve specific sets of customers plus allow
customers to build in their own, individual needs. It is not enough to be able
to "turn off" features with configuration switches (rather than resorting to
hard coding), which may help users but only further increases the complexity
of the code. For more information, see What's
Wrong With Application Software? Businesses Really Are Unique - One Size Can
Never Fit All. As a summary, finding the proper balance of modifications
versus bloat is subject to many factors, but the knowledge of what functionality
is already available is a critical starting point.
On the other hand, given that within a specific client size range and vertical industry, many renowned application packages are slowly but surely, reaching a functional parity (convergence), users might be better off by skipping the painstaking process of RFP preparation, staring confusedly at vendors' responses, and trying to figure out who has the most pluses regardless of the individual importance of the functionality criteria. It is better for organizations to focus on the handful of business objectives they need to achieve and the ways to measure their success.
is where the process-based part of the selection gig comes to shine. A full
list of functions cannot work unless they are organized into a path or business
process. For example, if the feature called "available-to-promise (ATP)" is
missing or not appropriate for your needs, the business process called "order-to-cash"
may be very difficult or impossible in your situation, given the planning step
will not work accordingly. To that end, the scripted scenario demonstration
phase of an enterprise application selection process is the perfect opportunity
to put candidate packages through their paces, and TEC urges users to exercise
this prerogative. However, instead of letting vendors take charge of the demo
and show you their dog and pony shows, insist on vendors unequivocally showing
you how their systems will help you achieve the desired objectives (see Demonstration
Post-Mortem: Why Vendors Lose Deals.
RFPs and selection tools typically focus on features and functions. The business process protagonists consider this focus old fashioned. However, users want and need an inventory or check list of the functions to understand if the business process will work. One always has to start from somewhere, and there is no better place to start researching enterprise software than from its functional and technical capabilities. Selecting enterprise software based on a business process approach can be better than a strict feature and function approach only when the user company has gone thorough the legwork ahead of time to determine 1) what functionality is required to enable each business process and 2) how important is each business process to enabling the companies' strategic objectives (i.e. relative importance of business processes) and how important is each item of functionality to enabling each business process (i.e. relative importance of the functional items that fall under each business process).
the most important take away from this analysis is the significance of buyers
researching technology vendors before determining the short list, while vendors
and VARs should research the viability of the opportunity beforehand. Issuing
a comprehensive RFI to a number of vendors is an important first step in the
selection process. Once the RFIs have been returned, analyze each RFI to determine
the strengths and weaknesses of each vendor as well as the relative importance
of each item on the RFI. This, proverbially harrowing exercise need not be that
dreadful, at least when leveraging ready-made repositories, as shown in CRM
Selections: When An Ounce Of Prevention Is Worth A Pound Of Cure, Selecting
PLM Software Solutions and Facing A Selection? Try A
Should you start at the top (business processes) or the bottom (functions)? You can start at either. If you know certain functions are very critical to your business (fatal flaws) then look at those functions first because without them, business processes are of no use. If you think your needs are covered by most products, start at the key business processes then investigate the key functions within those processes. Still, you must have the ability to unequivocally prove the fact that most of the solutions you are considering are on par regarding your needs. While the business process aspect is indisputably important, challenge the overly "process preaching" vendors to account for the functions inventory that comprises each process step.. Their readiness to accommodate you with both approaches would be a great sign and vice versa, beware of any vendor insisting on only one approach.
As a summary, the following are some of the main mutual benefits of all the parties being on the same page:
Upfront identification of issues and negotiation perspectives, which enable
more efficient and productive negotiations
Enabling the solution implementers to be more aware of the challenges
Enabling vendors to be aware of product gaps with client needs
Manage expectations of the implementation results to be realistic
Enable better implementation planning
Enable future project discussions between the vendor and client to be processed
more effectively, since past data is intact and in a form that is reusable
and can be updated easily.
To put vendors on notice that you expect a reasonable, demonstrable response, it helps to include a statement that the vendor's proposal will be an addendum to the signed contract. In case of any dispute (God forbid!), you will have to prove that the vendor did not conform to a list of several hundred functional specs built into the contract. Proving that at the level of certain business processes would be much more difficult owing to the more fluid nature of business processes.
no mistake—TEC does not expect anybody to acquire a crucial and costly piece
of technology based only on on-line research, however thorough it may be. Users
are therefore advised to conduct a thorough analysis of vendor strengths and
weaknesses in the following major areas: product functionality, product technology,
product TOC, corporate strategy, corporate viability, and corporate service
and support. Only by a diligent process of evaluation that includes a plethora
of other factors influencing the decision such as scripted scenario demonstrations,
site reference visits/calls outcomes (see Client
References - Still A Valuable Part of Vendor Selection?), product flexibility
(e.g., customizability, interconnectivity, data conversion, etc.) can users
hope to select an enterprise business system that will serve their organizations
and deliver the expected benefits.
are however, more of a soft, subjective nature, and require an actual encounter
with the software; this is where the human side will get the right of way over
machine in the above-mentioned human-machine combination. For more information,
Overview of the Knowledge Based Selection Process. Basing a decision only
on product functionality may result with buying a system that will soon become
obsolete. Advanced technology bolsters product flexibility and often can provide
tools that can circumvent the need for expensive modification (see Great
Product: Too Bad The Architecture Doesn't Fit).
But again, technology should not drive the business needs, rather it should be the other way around. Do you really need a sexy piece of technology that has missing functionality and will not cater to your business needs without significant modifications and system tweaking?
About the Authors
Jakovljevic is a research director with Technology Evaluation Centers,
Inc. (TEC), with a focus on the enterprise applications market. He has over
fifteen years of manufacturing industry experience, including several years
as a power user of IT/ERP, as well as being a consultant/implementer and market
analyst. He holds a bachelor's degree in mechanical engineering from the University
of Belgrade, Yugoslavia, and he has also been certified in production and inventory
management (CPIM) and in integrated resources management (CIRM) by APICS.
Thompson is a principal of Process ERP Partners. He has over twenty-five
years experience as an executive in the software industry. Olin
has been called "the Father of Process ERP." He is a frequent author and an
award-winning speaker on topics of gaining value from ERP, SCP, e-commerce,
and the impact of technology on industry. He can be reached at Olin@ProcessERP.com.
J. Strub has extensive experience as a manager and senior consultant
in planning and executing ERP projects for manufacturing and distribution systems
for large to medium-size companies in the retail, food & beverage, chemical,
and CPG process industries. Additionally, Mr. Strub was a consultant
and Information Systems Auditor with PricewaterhouseCoopers and an applications
development and support manager for Fortune 100 companies.
can be reached at JoeStrub@writecompanyplus.com.