Great Product: Too Bad The Architecture Doesn’t Fit

Introduction

Most potential customers understand that detailed scripted scenarios should be performed with software vendors who are on the short list of candidates, in order to evaluate the functional abilities of products being considered. Unfortunately, many companies neglect to investigate the technological underpinnings of these products during the evaluation phase. This can lead to the purchase of a product which is functionally excellent, but difficult (or impossible) to support in the customer's environment.

This document is designed to help craft a series of "Technical Architecture" meetings with vendors that can begin to explain the "how" of a vendor's solution, not just the final functional result. If the customer follows TEC's methodology for functional scripted scenarios and a vendor is still in the running, the next step must be taken to ensure a smooth product implementation and effective on-going support in the customer's environment.

Subject

Be Prepared to Discuss the Right Subject:

It must be stressed to the vendor that this phase of the software selection process is designed to evaluate technical (i.e., does the server run on Windows NT, is it a 2-tier or a 3-tier architecture) issues, not functional ones (i.e., how is payroll processed). The vendor should be provided with a list of questions in advance and given adequate time to prepare their responses. Someone in the organization should be designated to get clarifications for the vendor where required, before the on-site meeting.

Major Evaluation Criteria (with examples; detailed criteria are provided in TEC's selection model):

• Product Architecture
  
  
  • Examples: processing through application tiers, middleware and messaging components, and scalability.
    
    
• Data Protection & Restoration
  
  
  • Backup and restore methodologies, archiving.
    
    
• Security
  • User authentication and authorization, transaction and database security, directory services support.
    
    
• Tools
  
  
  • Diagnostics, session control, print spooling.
    
    
• Performance
  
  
  • Issues such as performance degradation when multiple application windows are opened on the same workstation, thread control between tiers of the application, network bandwidth and latency issues.
• Reporting
  
  
  • Support for ad hoc reporting, third party integration, and custom report design.
    
    
• Support for Client-developed Applications
  
  
  • Tools and control mechanisms, configuration management, methodologies.
    
    
• Vendor demonstration of selected features
  
  
  • The vendor should be tasked with demonstrating the most important technical features (i.e., demonstrate assignment of security to a user, monitoring application performance, etc.) and notified in advance and in detail as to what must be shown. For tips on how to make this process successful, see "Demonstration Post-Mortem: Why Vendors Lose Deals". This document discusses functional demonstrations, but the concepts remain the same.

Audience

Bring the right people:

The most important criteria for the success of a technical architecture software evaluation is the presence and participation of individuals qualified to hold the discussion. Once the proper subject areas have been determined, it is imperative that the customer provide a "jury" of individuals in the proper disciplines to ensure that subjects are properly covered. These individuals must be tasked with doing the background work to understand what the vendor is proposing, and fit that proposal into the "template" of their unique environment.

There need not necessarily be a single individual for each of these areas. Some employees will be able to speak to multiple subjects.

If both the customer and the vendor do not send qualified individuals to the meetings, this effort will be severely hampered.

Suggested Participants: (with examples of some roles they may fulfill)

• Overall "Process Administrator"
  
  
  • An individual responsible for the overall coordination of the "jury". This individual tallies ratings from the other participants and collates a collective score for each function point. This individual is usually also responsible for reporting progress to, and getting clarifications from, higher levels of management within the company.
    
    
• Network Administrator
  
  
  • To address issues such as bandwidth requirements, network protocols.
    
    
• Database Administrator
  
  
  • To discuss needs for specific databases (e.g., product only runs on Microsoft SQL Server).
    
    
• System Administrator
  
  
  • Discussion of platforms, disk space usage, CPU requirements.
    
    
• Security Officer
  
  
  • Examine implications of LDAP (Lightweight Directory Access Protocol) support, single sign-on, role-based access.
    
    
• Application Programmer
  
  
  • Discuss availability of vendor published API to enable custom enhancements/extensions.
• Web Programmer
  
  
  • Investigate web server and browser requirements, need for active server pages or other software dependencies, etc.
    
    
• Functional Representative
  
  
  • This individual may not understand much of the technical discussion, but is an invaluable aid in helping the group to understand the objective.

Summary

If everyone involved is engaged and the proper preparation has been done, technical architecture discussions can be extremely fruitful. Various architectural features of the products can be compared to each other and contrasted with those from other vendors. TEC has a Technical Architecture Practice with experience in addressing these issues and following up with vendors on unresolved questions. Some vendors have been surprised to see that customers are interested in "how things work", but soon realize that it is to everyone's advantage and that a full understanding by all parties will help ensure a smooth rollout, implementation, and on-going support of the product. A good "out of box" experience is best for all concerned.