Event Summary
Clickstream analysis, pop-up questionnaires, and focus groups are all useful ways to get information about a website. None is as good as hiring a usability professional to watch a large number of customers use the site. If you want to know how your site compares with your competition on some particular measure, like the ease of finding information or the convenience of placing an order, you'd also have to watch customers using those other sites. WebCriteria has an alternative. They have built a model describing how users behave on websites. The model is based on usability tests with live surfers, where measurements were made of how different web page features such as load time, number of links, and size and placement of graphics affected surfer behavior.
Their first product to combine these behavioral science studies with software agent technology, Site Analysis (originally named Max; see "Will Max Get Mad When He Surfs Your Website?"), scans a site and reports on its key characteristics as perceived by the model. The company has now introduced a second product, Task Analysis, which reports on the user's experience when performing and completing specific tasks on a website. The ratings that Task Analysis assigns agree with those of human surfers 79% of the time, making it a highly reliable tool.
WebCriteria defines user experience in terms of time and effort. Time is measured by such parameters as loading of text and graphics, page reading, navigation and entering information. Since Task Analysis works from a model of how much time users will spend on pages based on characteristics of the page the ratings are not dependent on the actual network conditions when Task Analysis evaluates a site. Measurement of effort involves such factors as number of pages and action, the number of distracting alternative links on the pages, the number of choices the user must make and the amount of information the user must supply.
WebCriteria's detailed reports are action oriented. They identify problem areas and suggest possible approaches. For example, one report page begins
"The current view shows pages sorted by influence with load time or access time measurements that have reached "suspect threshold" levels. The higher the influence, the more impact the page has on the rest of your site. For optimal effectiveness, improve pages with the highest influence first."
It then offers the following choices, which result in different presentations of the detailed data:
What do you want to do now? Work on the most significant accessibility issues on this site. Analyze page views with the worst access times. Analyze page views with the worst load times.
This solution-oriented approach is part of the value that WebCriteria delivers to customers. They not only provide the data, they also provide interpretation. In fact, the company has recently added to all of its services a careful review of the reports by a human analyst. The analyst highlights the most significant problems and points to best practices that will lead to improvements.
Task Analysis is offered in three different packages. With a One Time Report a customer buys a package of task evaluations. The minimum, five, costs $3500 and would typically be used to evaluate two or three different ways of performing some task - such as buying one item or two, or having a different shipping and billing address - and a few comparisons of competing sites. The next level is a Subscription. Starting at $11,000 a subscription provides regular tracking of key tasks for the site over a year. Finally, the "Full Service" program, starting at $30,000, augments the Subscription offering with detailed analysis and recommendations from WebCriteria's consultants; these analyses will take into account other sources of information such as clickstream data and measured performance.
Market Impact
WebCriteria argues persuasively that that have no true competition. Neither clickstream analysis firms, e.g. BizRate, nor even site-based questionnaires provide the kind of information that Site Analysis and Task Analysis do. The closest competitor may be a company called Vividence, which uses panels of surfers to evaluate websites. In any case, WebCriteria recommends that a website employ more than one kind of kind of tool to measure performance and usability. This is a recommendation that TEC endorses.
As website competition gets more cutthroat it becomes critically important to induce every possible visitor to your site to return - without of course offering more in benefits than those visitors are likely to spend. The next best way after free gifts to get visitors to return is to have them like the site, and studies show that the time it takes to perform tasks is a critical component of the overall rating of a site. Given the fairly low price of WebCriteria's entry-level offerings and the comprehensive nature of their high-end package they should find it fairly easy to increase their customer base.
User Recommendations
It is a pervasive problem in the industry that website managers can get stacks of data about their websites but little help in converting it into actionable information. The subtext of WebCriteria's product offerings is that the company focuses on providing concrete suggestions based on (modeled) user behavior on your site and those of your competitors'. With prices as low as $3500 it is hard to imagine any site that gets revenue from having users perform tasks like registration or purchasing on its site passing up a Task Analysis from WebCriteria.