Evaluating Enterprise Software - Business Process or Feature/Function-Based Approach? All the above, Perhaps?


Ever since the advent of business applications a few decades ago, selecting a piece of enterprise application software has proven to hardly ever be an exact science. Vendors' hype, consultants' "conflict of interest" and consequent bias, users' doubts and apathy, tediously long selection processes, and unclear decision rationales are some of the unfortunate watchwords for the selection practice so far. It has indeed been daunting for corporate IT buyers to discern the true capabilities, strengths and weaknesses, and degree of fit of a given enterprise application suite for their business environment. The fact that these solutions are not readily available from a local "Enterprise Applications R' Us" outlet (unless we are talking about the likes of QuickBooks for some "mom and pop" small office/home office [SOHO] businesses) and the propaganda that pervades vendors' endeavors to differentiate themselves, only serve to complicate matters (see Beware of Vendors Bearing Solutions). When making strategic IT acquisitions, buyer's project teams, inundated with an abundance of available products and technologies, have a difficult time translating the content of glitzy marketing brochures and grandstanding presentations into the deliverable products' capabilities.

This is Part One of a three-part note.

Part Two will discuss a business process-based approach.

Part Three will cover knowledge bases and make user recommendations

Prospects' Problem Overview

Prospective customers typically struggle with the following issues when selecting enterprise technologies:

  • For reasons unknown, users have a difficult time documenting what they do. This reluctance delays the selection process and the eventual benefits to be derived from the software. Unfortunately, this delay occurs at the early stages of a selection process, just when you are trying to gain momentum. Trying to break through this logjam requires a series of interviews, white board sessions, and flowcharting. While you could argue that this is a good learning experience, when time is of the essence, finding effective shortcuts can save the project and preserve the team's morale.

  • The selection project teams have no effective way to define their business requirements and thereby identify the critical vendor and product questions (criteria) necessary to successfully initiate the evaluation process. One possible option is to hire an outside consultant who will lead you through a number of interviews, meetings, workshops, etc. to determine your requirements. When you count the number of man-hours your team had to forsake to do this, the time it takes the outside consultant to come up to speed, and the hefty invoice for consultants' valuable hours, you may want an alternate approach. In a somewhat better case scenario, you might be paying a less exorbitant amount of money for a slightly tweaked request for information/proposal (RFI/RFP) document that has already been used many times before elsewhere.

  • When these criteria have eventually been pinpointed and submitted to (hopefully) the most appropriate vendors, project teams often have no objective ability to effectively prioritize the different criteria relative to one another. As a result, final priorities are often more the result of internal political agendas than true needs and requirements. Without having a valid statistical tool to keep these priorities in check and to conduct simulations of results after changing priorities, it is likely that some department's needs (e.g., accounting or the IT department) have, at the end of the day, been the unreasonably high contributor (e.g., well over 50 percent) of the total decision.

  • Project teams have no ability to obtain objective, independent, validated, and up-to-date data on the available vendor/product alternatives. Unfortunately, most project teams have no true ability to separate fact from hype, especially because its strategic technology selections are often either the first of its kind or the first in an extended period within a specific organization.

  • Even after the first pass at the criteria and vendor selection, changes in priorities and requirements are likely. Considering that this is an iterative process, responding to these changes without the benefit of a software tool can be a daunting task.

As a result, the vast majority of enterprise technology evaluations run over time and budget, and once selected, the majority of the implementations fail to meet functional, return on investment (ROI) and total cost of ownership (TOC) expectations (see The Joy' Of Enterprise Systems Implementation). At the same time, the number of companies, which are able to substantiate (quantify) the rationale of the decisions to go with some product in the past, is sadly negligible. A much higher number of decision makers rely solely on frivolous justifications like their gut-feel, executive mandate, or tiresome spreadsheet compilations without ability to discern the best solution (other than to, for those who have attempted some due diligence, arithmetically count the number of pluses and minuses, without taking into considerations their true importance). Reviewing the objective and logical path followed to arrive at a preferred vendor can disarm even the harshest dissenter to the selection process.

Furthermore, how many of these can produce the "hard" compelling rationale for switching to another, more viable vendor's product, following up a poor service track, demise, or an unfriendly acquisition of their current provider? Many companies have consequently been stuck with under-performing software products and dejected users, and are still unable to calibrate their systems to determine how far they are from the ideal solution for their business requirements. In other words, even after all the pain, many are unable to benchmark whether to maintain the status quo or try their luck that this time they will select another product that is the right product. The enterprise applications users' predicament has been duly covered at the TEC web site in the past (see The "Old ERP" Dilemma: Replace or Add-on, Application Erosion: Eating Away at Your Hard Earned Value, and Application Erosion: More Causes and Cures).

Vendors' and VARs' Side of the Predicament

Vendors, their VARs, and system integrators (SIs) are not in an easy situation either, despite their above-mentioned tendency to quite often oversell their products. On the vendor side, the challenge of educating the potential client of their offerings results in painfully long and costly sales cycles, painstaking and numerous responses to questionnaires time and again, and the potential for pursuing a mismatch opportunity, resulting in projects that can go terribly awry. Some may argue that this is just a cost of vendors doing business. These failed projects do not bode well for the vendor, since the sales cycle costs can only rise even more, and its reputation can suffer. The consequences can be much more severe for the client where it can, in extreme cases, lead to business failure.

Vendors will argue that, by implementing their software with little or no modifications, the client can take advantage of the best practices embedded in the software. However, unless this client has the structure and commitment to embrace these practices, havoc may prevail. Typically, clients do not foresee the impact of these changes on their organizations or estimate the internal resistance.

For implementers, the issue is similar: having inadequate fit-gap information for the implementation phase means an inability to properly plan and execute the implementation with an inevitably dreadful "scope creep," or for a consultant to assist the client in making proper technology decisions.

Referring back the vendor practice of selling "business processes" or "solutions," it is not your typical business model whereby the procurer of software and services must conform to the methodology of the seller. Clients prefer to present their requirements in a format that they are comfortable with. Clients are comfortable with defining their needs and letting the vendors document how they can satisfy these needs.

Often the users do not know why a particular product was selected in the first place (typically, when the system is eventually up and running and frustrated users start asking questions, the persons who selected the software are no longer there), while the vendors and VARs are not sure why they have won or lost some opportunities. Even winning serendipitously is not good, as it does not grant recurrence. Did we win because we had the best product, because we demonstrated it best, or maybe, to our delight and luck, because the Vendor X and Vendor Y were not invited to bid?

While it would be unfair to disparage the visionary and strategic level research that some analysts and IT research houses provide to vendors to help them determine their future direction (e.g., buying market spending forecasts, various surveys' findings, whether to embrace a services oriented architecture [SOA] and which industry standards, etc.), this by no means answers all the nitty-gritty questions vendors and VARs often wonder about. For example:

  • What and how many product functional and technological features, vertical industry or business process oriented, do we have to achieve to be the absolute frontrunner in the market segment or to beat (sometimes a dark horse competitor) Vendor X in a particular face-off? Does the newly released module or the latest product upgrade make us significantly more eligible to compete in the market segment?

  • Are we still one of the leaders in a certain industry, merely because of the number of installations? If yes, then how many of these have come from, for example, our accounting or human resource (HR)/payroll solution only as opposed to from our sharp industry solution per se? Were we perhaps the first one to tackle the market segment; if so, is our solution still stronger than a newcomer's one, with fewer customers but with a formidable offering we do not know much about yet?

  • Should we compete (and commit to the non-refundable hefty expenses and resources involved) in a selection in a certain geographic region against a large vendor that certainly has more functionality, but is less flexible than we are? What other trade-offs should we present to the customer knowing our strengths and weaknesses, and will they fly? Do we have any more concrete documentation to back up our claims other than an Analyst A's fluffy report that arbitrarily praises our product (the opponent will likely have a similar report from Analyst B)? What about competing against a dark horse vendor we have never seen before, but the market seems to be raving about its product?

  • How best can we target a struggling or defunct competitor's stranded install base? How, black on white and in an intuitive manner, do we prove that our solution would be just what the doctor ordered for its disconcerted customers?

  • How do we increase our reference index to attract new clients? Is our "hot line" support sufficient to resolve issues in a forty-eight hour window? Do we effectively communicate known issues instead of allowing clients discover them on their own?

The actionable, quantifiably substantiated, and in no uncertain terms answers to the above questions, while arguably obtainable from traditional analyst houses, would likely be exorbitantly expensive, given the annual subscription fees for much less granular information and service these firms traditionally charge.

Therefore, despite apparent adversarial relationships, all parties (i.e., customers, vendors, and consultants) have meanwhile realized the benefits and need for thought and research to build data and process information in a meaningful context, which takes time and costs money and human resources occupancy for all parties going through the selection process. But without spending time, thought, research, and money there is increased business risk to all, unless there is an existing inexpensive solution to provide a structured, repeatable process for evaluating technology solutions and the vendors that provide them.

Most business managers, whether within vendors, prospective clients or implementers/resellers, have long yearned for an enabling repository system to minimize project risk for all sides of technology utilization. For instance, by quickly narrowing products down to a shortlist based on functional and technical features, vendors and VARs benefit from not pursuing unfruitful clients. Clients benefit because the short list is usually reduced to a manageable size. Finally, everybody benefits because this unreasonably costly and tedious phase can be streamlined.

How Has It Been Done So Far?

To that end, owing to learning from the past experiences and to the help of specialized selection service providers, selecting an enterprise package has, to a degree, become a routine occurrence in the life of an IT organization. What makes this task challenging is the complex nature of software being evaluated relative to the scarcity of straightforwardly suitable solutions owing to their expansiveness in scope. The approach, fortunately, can be independent of these factors. These factors may mean that more time is needed to complete the project but the basic steps can essentially be the same. The emphasis and priority placed on specific steps, however, can change to suit the situation and package being considered (see Software Selection: An Approach and Ways of Finding Software Vendors: The Pro's and Con's).

You should expect a specialized selection service provider like TEC (www.technologyevaluation.com) to cover a horizontal functional scope such as enterprise resource planning (ERP), supply chain management (SCM), warehouse management systems (WMS), customer resource management (CRM), product lifecycle management (PLM), enterprise asset management (EAM) and so on. Then, typically, a service should allow you to:

  • Provide a list of commonly found features and functions by department, process, and module, thereby minimizing the aforementioned user reluctance to complete this necessary documentation.

  • Rank the importance of function areas such as financial management, production, and customer service.

  • Select the required lower-level functions and features by the above areas (i.e. financials, production, customer service), while unneeded ones are grayed out.

  • Rank the importance of functions within each area.

Once this is done, the service should be able to match your needs and requirements against its database of vendors, and then suggest potential vendors appearing to meet the business needs of your company. Obviously, this should be a repetitive and iterative process that is refined as rankings are changed and weights are reassigned. The process is continued until the project team is confident that the model reasonably and accurately portrays the company's business practices and needs.

The bottom line is easier for users to define their needs, requirements, and improvements if they are provided with a potential list. It could then be a simple of matter of selecting from the list. The theory is that you see what you don't want or need, you then must know what you do want and need.


Prior to Y2K many IT shops had suffered from years of budget neglect. Technology had not been upgraded; software processes had continually been held together with Band-Aids and patches. With the potential, but unrealized, repercussions of Y2K, the mentality was to replace, not rewrite. Under this scenario and the above assumptions, looking at software from a business process perspective can make a lot of sense. Your current software environment is untenable, indefensible, and unmaintainable. Replace it with a vendor's solutions and best practices. A persuasive and logical argument.

However, if your organization does not fall into this state of disrepair, it may make little or no sense to completely disrupt your software environment and the users that it serves. In this case, you would be best advised to look cautiously at the features and functions you want to preserve, improve, or replace. Under this scenario, starting with a laundry list of capabilities and using a selection service makes prudent sense since it allows you to quickly develop a short list of vendors who can best proximate your needs and should be evaluated further.

A software selection service can save you time and money. It does not, nor should it, select a vendor at the push of a button.

This concludes Part One of a three-part note.

Part Two will discuss a business process-based approach.

Part Three will cover knowledge bases and make user recommendations

About the Authors

Predrag Jakovljevic is a research director with Technology Evaluation Centers, Inc. (TEC), with a focus on the enterprise applications market. He has over fifteen years of manufacturing industry experience, including several years as a power user of IT/ERP, as well as being a consultant/implementer and market analyst. He holds a bachelor's degree in mechanical engineering from the University of Belgrade, Yugoslavia, and he has also been certified in production and inventory management (CPIM) and in integrated resources management (CIRM) by APICS.

Olin Thompson is a principal of Process ERP Partners. He has over twenty-five years experience as an executive in the software industry. Olin has been called "the Father of Process ERP." He is a frequent author and an award-winning speaker on topics of gaining value from ERP, SCP, e-commerce, and the impact of technology on industry. He can be reached at Olin@ProcessERP.com.

Joseph J. Strub has extensive experience as a manager and senior consultant in planning and executing ERP projects for manufacturing and distribution systems for large to medium-size companies in the retail, food & beverage, chemical, and CPG process industries. Additionally, Mr. Strub was a consultant and Information Systems Auditor with PricewaterhouseCoopers and an applications development and support manager for Fortune 100 companies.

He can be reached at JoeStrub@writecompanyplus.com.

comments powered by Disqus