10 Tips for Checking Your Software Implementer's References

Your organization has selected to purchase an enterprise resource planning (ERP) software—or any other type of software for that matter. The next step is to find the software provider that will deliver and implement the software for you. This can be the vendor, an independent software vendor (ISV), or an authorized reseller, often called a value-added reseller (VAR). Finding the right service provider for your organization is a challenging feat. Oftentimes, a positive reference from a company that has already performed software implementation with a selected VAR or software vendor seals the deal. But as references have the potential for subjectivity and bias, you need a standardized approach to assessing the provider’s performance.

Although services to be delivered are usually described in detail in service level agreements (SLAs), there are factors that can’t be directly measured or estimated, which my colleague Jamal Rahal talks about in his recent blog post. For instance, level of involvement from top management, or the VAR’s employees’ attitude, dedication and loyalty, knowledge level, and commitment to the implementation project—these are all factors with a very subjective tone that can hardly be covered or measured in any formal agreement, but that can play a big role in how successful the provider was in fulfilling its commitments.

Many questions typically covered in SLAs are relevant here and should be asked during a reference check:

  1. How was the scope of the work defined?

  2. How would you rate the service provider’s performance?

  3. Were problems resolved in a reasonable manner?

  4. Were intellectual property rights and confidentiality rules respected?

In addition, when requesting an assessment of a service provider through a third-party reference, companies should refer to some generic criteria with which to compare and evaluate different service providers. Below is a list of 10 formalized and standardized criteria I have created, based on my experience as a research analyst with Technology Evaluation Centers (TEC), in an effort to create a realistic understanding of the quality of the services delivered to the reference company. These criteria can be used in any type of business area or industry.

They are as follows:

  1. Delivery of services provided as promised, as planned, and in a timely manner. If this has not been done, then it is important to know the reasons why and the general or situational factors that contributed to it, as this may have implications for your implementation project.

  2. Services provided as promised, without necessitating the purchase of additional services. Various system modifications, customizations, data migration, or adjustments may be included in the SLA, up to certain point, but any additional work may require extra fees. You need to know what was covered and what wasn’t, and why.

  3. Service provider’s ability to meet its commitments—to reach project milestones and project completion in a timely fashion and within the pre-established timeframe. The provider’s inability to work extra hours and night shifts, and the presence of delays all point to the vendor failing to fulfil its commitments.

  4. Help desk response time and usefulness. The faster the response time, the better, of course—but the response also needs to be of high quality and relevance to the issue at hand.

  5. Provider’s ability to predict undesirable events during the implementation based on knowledge of the software, client’s business, and experience. The absence of emergency situations, unexpected discoveries, project delays, and rush work all indicate that the provider has enough knowledge and practical experience to do the required work—and thus avoid such events.

  6. Knowledge level of the provider’s staff. It is obvious that consultants involved in the project have a different level of knowledge and experience, but that should not be an impediment to project performance. It’s also imperative to quickly reach an agreement to replace a consultant at any point in the project if required.

  7. Quality of documentation provided. The project and software product documentation should be written in a way that end users can easily understand. This documentation should be provided in full to the client.

  8. Quality of training provided. This is related to the previous point and describes how easily and quickly the client’s employees were able to learn the software from the live training sessions and documentation provided.

  9. Ability to provide promised resources. VARs, in fact, can be involved in projects with different clients simultaneously but should be able to plan and schedule the necessary human resources (HR) personnel without compromising the quality of any given project.

  10. Accuracy of the provider’s estimates for the project costs and required resources. This point attests to the service provider’s ability to accurately estimate, plan, and schedule, as well as take overall responsibility for the completion of the project. For example, the closer the estimate is to the final tally, the better the provider knows and understands the client’s business.

The criteria above will provide you with a formalized analysis and assessment of the potential service provider’s overall level of commitment and responsibility to an implementation project. And certainly, the more criteria you have, the better visibility you’ll gain on your prospective contractor, and, eventually, the probability of the success of your own project.
comments powered by Disqus