Software Evaluation, Selection, and Procurement

  • Written By: Paul Winandy
  • Published: February 11 2003


A 1995 Standish Group research study based on over 8000 software implementation projects found that nearly one third of the projects were canceled before completion. And over 50% of the projects cost nearly twice their original estimates. The study further concluded that only 16% of all software projects are completed on time and on budget (only 9% for large companies)1.

With these grim statistics firmly in hand, a plethora of books, white papers, and articles have been written on implementation success factors and software implementation methodologies.

While these publications offer sound and sage advice, the problems remain and the project failure statistics have not improved.

One area receiving little scrutiny, however, is the software evaluation, selection, and procurement processes employed by most organizations. This article reviews common problems associated with software evaluations, and introduces several ideas for improvement.

This is Part One of a two-part article. Part Two will make Recommendations for Improvement.

1 - Standish Group International, Inc., The CHAOS Report, December 1, 1995.

The Capital Investment Paradigm

Financially, GAAP requires companies to account for most software license and implementation costs as capital investments. This makes perfect fiscal sense, as the software lifecycle is long and depreciating the investment over multiple years is sound financial practice.

Unfortunately, the software procurement process does not follow a similar, long-term capital investment approach. Instead most companies handle software purchases like departmental operating expenditures. This flawed paradigm is the root of many implementation failures and the cost overruns cited above.

To illustrate the capital investment approach, let's use the example of production machinery for a manufacturing firm. When considering an investment in new machinery, multiple departmental teams with varying requirements participate in the decision making process. Typical activities might include:

Team Activity
Manufacturing Management

Build business case for investment in new machinery

Investigate alternatives to new machinery purchase (i.e. outsourcing, continued use of existing machinery, etc.)

Investigate the service and warranty support provided by the candidate vendor(s)

Investigate the financial viability of the candidate machinery supplier(s)

Engineering Test the machine to confirm it will produce the necessary product within tolerance
Factory Operations Test the machine to make sure it fits within the allocated floor space, is easy to operate, and will improve productivity as expected
Quality Assurance Test actual runs of product through the new machine to confirm the error rate is within requirements
Customers Manufacturing process changes may need to be reviewed by external customers to ensure they conform with agreed to specifications
Suppliers Raw materials will need to be confirmed on the new equipment and possibly alternative suppliers must be evaluated
Procurement Negotiate final contract terms
Finance Arrange for acquisition of needed capital

Other forms of capital investment (such as facilities or real estate) generate similar analysis and due diligence activities from a variety of internal and external teams. This model recognizes the fact that a capital investment must serve the organization well for multiple years. Spending time and effort up front to ensure a smooth transition to the new machinery (facility, etc.) is simply good business practice.

Software Evaluation Challenges

Most companies do not treat software procurement in a similar manner. A series of common flaws can be found in most selection processes. The following table outlines the most common software evaluation and selection flaws, and the associated risk.

Flaw Leads to... Project Risk
Insufficient time devoted to evaluation and selection
  • Ill informed decisions

  • Poor software "fit" against business requirements
  • Improper expectations on implementation cost, time, resources, etc.
  • Erosion of project support
  • Increased user resistance
No business case
Poorly defined business case
  • Inaccurate projection of:
    • Costs
    • Benefits
    • Alternatives
    • Risks
    • Assumptions
    • etc.
  • Cost overruns due to poor planning
  • Lack of business benefits
  • Lack of "Ownership" of each individual benefit
  • Erosion of project support
Affected departments not part of selection team
  • One-sided evaluation of software fit
  • Unidentified software gaps
  • Poor software "fit"
  • Cost overruns due to unplanned work to resolve new software gaps
  • Erosion of project support
  • Increased user resistance
Incomplete or inaccurate set of business / technical requirements
  • Unidentified software gaps
  • Inability to sufficiently "test" the system
  • Cost overruns due to:
    • Unplanned work to identify new requirements at a later date
    • Unplanned work to incorporate additional functionality
    • Unplanned work to perform additional testing
    • Erosion of project support
    • Increased user resistance
Insufficient testing of candidate solutions
  • Unidentified software gaps
  • Unidentified integration issues
  • Poor software fit
  • Cost overruns due to:
    • Unplanned work to resolve new software gaps
    • Unplanned work to build interfaces
    • Erosion of project support
    • Increased user resistance
Insufficient verification with external customers/suppliers
  • Customer/supplier discontent with new system
  • Lost sales
  • Lowered customer service
  • Supplier cost increases
Insufficient due diligence of software vendor (i.e. references, viability, future development, support, etc.)
  • Poor support
  • Lack of future functionality for upgrade, enhancement
  • Increased total cost of ownership

The software evaluation and selection process builds the foundation for the entire implementation project. A foundation that includes one or more of these flaws will inevitably crack under the pressure associated with a mission-critical software implementation. A structured and disciplined software evaluation and selection approach will mitigate this risk and establish a solid foundation from which to move forward. The following section outlines our recommended guidelines for such an approach.

This concludes Part One of a two-part article. Part Two makes Recommendations for Improvement.

About the Author

Paul Winandy is a founding partner of Spinnaker Management Group (, a management consultancy guiding small and mid-sized businesses along the entire journey from strategy creation through technology implementation. Mr. Winandy has over 15 years experience transforming organizations through business process and technology solutions. He specializes in the planning, selection, and implementation of mission-critical ERP and supply chain related systems. Mr. Winandy has assumed project leadership roles on several major strategy and systems implementation projects for multi-national manufacturing and distribution enterprises in industries such as semiconductors, CPG, food & beverage, wholesale distribution and transportation

comments powered by Disqus