How Some ERP Vendors Demonstrated - Warts And All Part 2: Results

Overview 

The subject of this case study is the synopsis of a crucial step within every software selection process - finalist vendors' scripted scenarios software demonstrations; this particular series of events took place in February/March 2001. The importance of this milestone in any software selection undertaking has been widely publicized (for more information, see An Overview of the Knowledge Based Selection Process). Demonstrations can and should be grueling for both vendors and users, as they are the only way to discern how a software application behaves under real world expectations before any firm commitment and point of no return happens.

The company involved in this case study embarked on the selection of an ERP/business applications package that would satisfy its needs and return maximum business value. TEC was retained as a neutral process facilitator with an intimate knowledge of the market/vendors and no vested interest (e.g., future system integrating opportunity) in the outcome of the selection.

About this Note:

This is a two part note with Part One covered the Scripted Scenario Demonstrations. Part Two discusses the specific results of the demonstrations with recommendations for users facing a similar selection process.

Rating of Vendor Preparation and Performance 

Here we examine the preparation and performance of each vendor, giving each a general rating. The categories we examine are:

• Show vs. Tell
  
  
• Preparation of Script Items
  
  
• Ability to Follow Scripts or Mapping Document
  
  
• Ability to Finish During the Allotted Time

Show vs. Tell

Preparation of Script Items

Ability to Follow Scripts or Mapping Document

Ability to Finish During the Allotted Time

Results and User Recommendations 

The moral of the story should not be that a dazzling demonstration ability only will mean securing a business - a plethora of other factors influence the decision, such as the product architecture (for more information, see Great Product: Too Bad The Architecture Doesn't Fit), total cost of ownership (TOC), site reference visits/calls outcomes, etc. For these reasons, the winner of the entire selection process will not hereby be disclosed.

Conversely however, a bad demonstration performance will very likely seal the vendor's fate. Clients, as a rule, see in the vendor's diligence in preparing for the demo an indication of their future service and support experience with the vendor. Even large, successful enterprise applications vendors can fail when it comes to product demonstrations. Lack of preparation is the usual culprit, but vendors are not solely to blame. A diligent approach to preparing scripted scenario demonstrations by all parties involved (vendors, users, and the selection facilitator), particularly in terms of the demo's scope feasibility, will ensure that vendors convey the message and users obtain the insights they need to make a decision.

Based on the above analysis of data in the decision model, our client was able to clearly differentiate the vendors and bring out the strengths and weaknesses of each. Some other advantages of performing a thorough, documented scripted scenarios process are worth mentioning:

• The best-placed vendor still failed to satisfy almost 25% of the criteria straight away. Such shortcomings, while disconcerting, can still provide considerable leverage during final negotiations if managed skillfully.
  
  
• The discrepancy in rankings of vendors after responding to RFP and after the scripted scenarios (see Figures 1 & 2 in Part 1) indicate that one vendor's "Yes" may be less valid than the other vendor's "Yes, but", and the importance of putting the products through their paces. However, a neutral expert's presence is preferable as to discern the substance from a mere ability to demonstrate.
  
  
• When more than one vendor ranks close within a given set of areas, the decision model provides the supporting material required to justify further investigations. These include scripted scenario demonstrations follow-ups and client reference visits, both of which were utilized by the manufacturer in this engagement.

Finally, this case brings out an important fact of software selections that is often overlooked - the best solution almost always involves compromise. In this engagement, having the results in the selection tool provided a means for setting expectations among project team members and senior management. Such disclosure at an early stage can prevent disappointments later in the selection process.

Note: This selection involved the use of TEC's decision support tool ERGO 2001. For more information on ERGO 2001 see ERGO 2001 IT Evaluation Tool).

If your company is interested in TEC in knowledge based selection contact sales@technologyevaluation.com.