Make smart and accurate
software selection decisions
Podcasts, Webinars, and Videos
Interactive Case Studies
ERGO Decision Support System
Private Label Partnerships
TEC Case Studies
Software Evaluation Reports
Meet TEC's Experts
News and Press Releases
Working at TEC
Partner with TEC
Behind the Scenes of TEC’s Vendor Showdowns
Behind the Scenes of TEC’s Vendor Showdowns
September 3 2008
TEC’s Vendor Showdown series is the
most popular regular feature
that we run here at TEC. So far, we’ve run quite a few Showdowns, and usually things go smoothly, but sometimes there’s more that goes on than meets the eye—and then it’s usually problems, as you’ll see in a moment. I should know, I’m the person who writes TEC’s Vendor Showdowns.
For those who've never seen one,
Showdowns compare three software solutions
(it could be ERP, or CRM, or BI, or HR, or any of the over 20 different kinds of software that we cover) on the basis of pure functionality. We publish a series of charts based on the latest RFI data we collect from the three vendors, provide a bit of commentary, and let the reader see who placed first, second, or third, both overall and by main modules. (Here’s a link to the latest
ERP Vendor Showdown
featuring Infor, Epicor and Lawson, and you’ll see what I mean.)
Showdowns give our readers an
opportunity to discover new vendor solutions
, re-acquaint themselves with once-familiar ones, and come across solutions they find interesting enough to evaluate further or include in their shortlists. Showdowns give vendors the chance to expose their solutions to an interested group of people, some of whom may not even be aware that such and such a vendor offers this or that type of software solution.
It sounds pretty simple so far, but sometimes it doesn’t work out that way.
Firstly, Showdowns were never intended to be the final word on how good a piece of software is, or whether it’s better or worse than its competitors. Showdowns only provide
rough, overall, rankings
based on a purely quantitative calculation of how each vendor solution supports a long list of features and functions (is this or that feature or function supported straight out of the box, with a third-party add-on, in a future release, and so on.)
do not take into account
the special needs of your organization, or anyone else’s, which is instrumental in how one determines which is the best software solution—and we take great pains to point all of this out in the accompanying text. But inevitably, some people miss the point—or fail to read the text that accompanies the charts—and send us nasty emails telling us that this or that piece of software is really the best, and how is it that we were too ignorant to realize it. Well, the results are based on information we poll from the vendors, and we can only hope that people who are inclined to send us an email missive take the time to read the accompanying text first.
Secondly, we like to give the vendors a heads up before we actually publish the results, so that they’re not taken by surprise. Well, this isn’t likely to surprise anyone, but while the ‘winning’ vendor is inevitably happy with the outcome, the ‘losing’ vendors are always a lot less than delighted. And, of course, I understand this.
Vendors are highly competitive
– it’s built right into their DNA, and that’s partly what’s helped make the software industry the tremendous success that it is. But as soon as the vendors get their advance look at the Showdown, the emails start to fly back and forth, with some vendors questioning the veracity of other vendors’ data, and other vendors just basically crying foul. Ouch. The problem, of course, is that if there is a perception that if somebody ‘wins’, then there is a perception that somebody else must correspondingly ‘lose’. And the ‘losers’ are understandably never happy.
But in truth,
there are no ‘winners’ or ‘losers’
in TEC Showdowns, simply solutions that are designed with differing purposes and end users in mind—no software solution fits everyone’s needs, and no software solution is perfect. Not only that, it is often only a very few percentage points that separates the leader from the second or third ranked solution. One solution
that we hope will make everyone happy is to feature only TEC certified solutions—which means that all of the vendor RFI data in the Showdowns will have been certified to be accurate by our TEC analyst team in conjunction with the vendor in question. That should help prevent vendors from being concerned that their competitor’s data in a Showdown is anything less than completely accurate.
Thirdly, we’ve had a couple of Showdowns that never even saw the light of day. I like to think of them as
Showdowns that Almost Were
. One was an LMS Showdown with two vendors (it was going to the first of a new format that features two vendors instead of three). It turned out that both vendors felt their data was either slightly out of date or inaccurate or both, and in the end, the best solution was to put this Showdown in the vault. Will it ever see the light of day – I doubt it. We will be publishing an LMS Showdown soon, but I can’t imagine it will be with the same data.
The other was a Showdown that we actually started to run, but quickly had to
pull down off the site
. Everything went wrong on this one—we interpreted the data incorrectly, we published data that we weren’t supposed to, and we inadvertently made it possible to identify the client (this one was based on a real case study) and so on, till we managed to displease not only some of the the losing vendors (there were actually five in the running this time), but the winning vendor as well, and, if I recall correctly, the client too.
I suppose, in the end, that all of this is just the price of success. Or proof that
is alive and well. But either way, keep watching—there will definitely be more Showdowns to come because you, our audience, likes them—and the vendors usually do too. And in the end,
that’s what really matters.
comments powered by Disqus.
comments powered by
Interested in a better way to make software decisions?
Give us a call now: 1-800-496-1303 ext:404
Software Requirements Sets and Comparison Reports
Click here to leverage the experience of our 360 industry perspective