It’s All About User Experience But, How Can We Measure User Experience?
Clickstream analysis, pop-up questionnaires, and focus groups are all
useful ways to get information about a website. None is as good as hiring
a usability professional to watch a large number of customers use the
site. If you want to know how your site compares with your competition
on some particular measure, like the ease of finding information or the
convenience of placing an order, you'd also have to watch customers using
those other sites. WebCriteria has an alternative. They have built a model
describing how users behave on websites. The model is based on usability
tests with live surfers, where measurements were made of how different
web page features such as load time, number of links, and size and placement
of graphics affected surfer behavior.
first product to combine these behavioral science studies with software
agent technology, Site Analysis (originally named Max; see "Will
Max Get Mad When He Surfs Your Website?"), scans a site and reports
on its key characteristics as perceived by the model. The company has
now introduced a second product, Task Analysis, which reports on the
user's experience when performing and completing specific tasks on a website.
The ratings that Task Analysis assigns agree with those of human surfers
79% of the time, making it a highly reliable tool.
defines user experience in terms of time and effort. Time is measured
by such parameters as loading of text and graphics, page reading, navigation
and entering information. Since Task Analysis works from a model of how
much time users will spend on pages based on characteristics of the page
the ratings are not dependent on the actual network conditions when Task
Analysis evaluates a site. Measurement of effort involves such factors
as number of pages and action, the number of distracting alternative links
on the pages, the number of choices the user must make and the amount
of information the user must supply.
detailed reports are action oriented. They identify problem areas and
suggest possible approaches. For example, one report page begins
current view shows pages sorted by influence with load time or access
time measurements that have reached "suspect threshold" levels. The higher
the influence, the more impact the page has on the rest of your site.
For optimal effectiveness, improve pages with the highest influence first."
then offers the following choices, which result in different presentations
of the detailed data:
do you want to do now?
- Work on the most significant accessibility issues on this site.
- Analyze page views with the worst access times.
- Analyze page views with the worst load times.
solution-oriented approach is part of the value that WebCriteria delivers
to customers. They not only provide the data, they also provide interpretation.
In fact, the company has recently added to all of its services a careful
review of the reports by a human analyst. The analyst highlights the most
significant problems and points to best practices that will lead to improvements.
Analysis is offered in three different packages. With a One Time Report
a customer buys a package of task evaluations. The minimum, five, costs
$3500 and would typically be used to evaluate two or three different ways
of performing some task - such as buying one item or two, or having a
different shipping and billing address - and a few comparisons of competing
sites. The next level is a Subscription. Starting at $11,000 a subscription
provides regular tracking of key tasks for the site over a year. Finally,
the "Full Service" program, starting at $30,000, augments the Subscription
offering with detailed analysis and recommendations from WebCriteria's
consultants; these analyses will take into account other sources of information
such as clickstream data and measured performance.
WebCriteria argues persuasively that that have no true competition. Neither
clickstream analysis firms, e.g. BizRate, nor even site-based questionnaires
provide the kind of information that Site Analysis and Task Analysis do.
The closest competitor may be a company called Vividence, which uses panels
of surfers to evaluate websites. In any case, WebCriteria recommends that
a website employ more than one kind of kind of tool to measure performance
and usability. This is a recommendation that TEC endorses.
website competition gets more cutthroat it becomes critically important
to induce every possible visitor to your site to return - without of course
offering more in benefits than those visitors are likely to spend. The
next best way after free gifts to get visitors to return is to have them
like the site, and studies show that the time it takes to perform tasks
is a critical component of the overall rating of a site. Given the fairly
low price of WebCriteria's entry-level offerings and the comprehensive
nature of their high-end package they should find it fairly easy to increase
their customer base.
It is a pervasive problem in the industry that website managers can get
stacks of data about their websites but little help in converting it into
actionable information. The subtext of WebCriteria's product offerings
is that the company focuses on providing concrete suggestions based on
(modeled) user behavior on your site and those of your competitors'. With
prices as low as $3500 it is hard to imagine any site that gets revenue
from having users perform tasks like registration or purchasing on its
site passing up a Task Analysis from WebCriteria.