How to Measure Customer Satisfaction (顧客の満足度を測るには)

Page 1 of 5

Project-based organizations place a lot of emphasis on customer satisfaction, and rightly so, as customer satisfaction is the key for improving these companies' internal processes¹. A customer satisfaction rating (CSR) is often obtained through a questionnaire—the customer satisfaction survey (CSS). This method, however, suffers from the drawback of² customers likely being emotionally influenced while filling out these questionnaires.

1. intérnal prócesses: 社内の業務プロセス
2. suffers from the dráwbàck of: ～の難点がある、～を欠点としてもつ。ここでは、顧客満足度調査 (CSS) だと、回答者は感情に左右された (emotionally influenced) 回答をしてしまう傾向にあるという短所を指摘している。

Naomi Karten, an expert on the subject of customer satisfaction (www.nkarten.com), states in her seminar Tales of Whoa and the Psychology of Customer Satisfaction: "People tend to rate service higher when delivered by people they like than by people they don't like.¹" Karten also goes on to describe what one can do to be "likable²." More often than not, Karten contends³, the customer satisfaction survey (CSS) rating received from the customer represents perceived feedback rather than impartial feedback⁴.

1. People tend to rate service higher when delivered by people they like than by people they don’t like.: higher ～ than ～ という比較の構文で、by people they like と by people they don’t like を比べている。「気に入らない相手にしてもらったサービスより、気に入った相手にしてもらったサービスの方を高く評価する傾向がある。」
2. líkable: 好感がもてること、好ましいこと
3. conténd: 熱心に主張する、強く主張する　
4. percéived feedback rather than impártial feedback: 中立的、客観的 (impartial) というより、感情なども含む回答者の認識 (perceived) によるフィードバック(回答)

This is not to say that companies do not get any value from customer-filled customer satisfaction rating (CSR) forms. But they must recognize that responses can be emotionally based, and that the customer is not one person, but an organization—meaning multiple people. While so, only one person represents the organization and fills out the survey. Would this person consult all concerned before filling it out? Ideally, he or she should, but often, he or she will not.

This gives rise to¹ the need for a way to compute a customer satisfaction rating (CSR) based on internal data²—data that is free from bias³ and that gives a realistic metric⁴ on customer satisfaction.

1. gives rise to: ～を起こす、生み出す、生じさせる
2. intérnal data: 社内の業務記録データ
3. free from bías: 偏りのない
4. realístic métric: 現実的な評価基準

Why Should We Measure Customer Satisfaction with Internal Data?

Consider the following three scenarios:

1. scenário: ある状況の想定、シナリオ。ここでは、満足度の調査に回答する顧客企業の回答者の違いによって予測される3つの異なる状況を紹介している (1. 事実にもとづいて忠実に回答できる社員が回答する場合、2. ごく普通の人間であるため、ぴったり正確な判断ができず、実際よりも低い評価をしてしまう社員が回答する場合、3. 同じくごく普通の人間であるため、ぴったり正確な判断ができず、実際よりも高い評価をしてしまう社員の場合)。

1. The customer is pragmatic and not swayed by influences like the recency factor and the one-incident factor, prejudices of any kind, poor judgment, or personal stake¹. This customer keeps meticulous records of the project execution and is expert at data analysis. While it may be rare to have such a customer, his rating is likely a true reflection of the vendor's performance.




1. pragmátic and not swayed by ínfluences like the récency factor and the one-incident factor, préjudices of any kind, poor judgment, or personal stake: 近況 (recency) や一時的な出来事 (one-incident)、偏見 (prejudice)、評価の甘さ (poor judgment)、個人的利害関係 (personal stake) などのような要素による影響を受けて (回答に対する) 判断が揺らぐ (sway)ことがなく、現実的 (pragmatic) である keeps metículous records: 細部まで行き届いた記録を取る


2. The customer is an average person. His rating is influenced by some of the factors mentioned in the first scenario. Let us assume that he rates the vendor's performance as poor. If this low rating (which is biased) were accepted, the personnel involved in the project execution would also receive low ratings in the organization as a result. They might, in turn, receive lower hikes¹ (salary increases) and bonuses, if any at all. This would demotivate² these workers, as it is possible that they in fact did a fairly good job and merit a better rating.




1. recéive lower hikes: hikesとは、給料 (物価のこともある) の引き上げを指す。したがって、lower hikeで、給料の引き上げ率の低下
2. demótivate: (労働) 意欲を失わせる


3. The customer is an average person. His rating is influenced by some of the factors mentioned in the first scenario. Let us assume that he rates the vendor's performance as high. As a result, the personnel involved in the project execution might receive better hikes and bonuses. Such a situation would further demotivate the personnel from the second scenario.

Scenarios two and three give rise to the phenomenon known as "rewarding the underperformers and punishing the better performers¹"—a disastrous situation² for any organization. An impact even more disastrous is that the organization does not have a realistic picture of how satisfied its customers really are. In such a situation, any efforts to improve customer satisfaction would be taken in the wrong direction.

1. rewárding the underperfórmers and púnishing the better perfórmers: 標準よりも成績の悪い者に報酬を与え、標準よりも成績の良かった者を処罰する (上述シナリオの2番や3番のケース)
2. a disástrous situátion: 悲惨な状況

Aspects Critical to Customer Satisfaction
(顧客企業の満足度に関連した重要な要素)

I have been using the following method to compute a customer satisfaction metric, based on internal data, in all the organizations to which I have provided consulting services. I developed this system through reverse-engineering¹ of the vendor-rating metric that manufacturers use to rate their suppliers. The method is based on the five following parameters I believe are critical to customer satisfaction, which are tangible aspects that can be measured objectively².

1. revérse-enginéering: リバース・エンジニアリングといって、ソフトウェアやハードウェアを分解、解析することによって、その仕組みや技術を明らかにすること。ここでは、筆者は、製造業者の使用する仕入先の評価基準 (vendor-rating metric) を分類し、分析することにより、顧客会社の満足度測定システムを開発した、と述べている。
2. tángible áspects that can be méasured objéctively: 客観的な測定ができる具体的な(tangible) 要素

1. Quality(製品の品質)
Quality comes first. The dictum "customers forget the delays but not the quality"¹ aptly² states the value of quality. Furthermore, customers forget everything else if—and only if—the quality delivered is superb.

1. The díctum "customers forget the delays but not the quality": 「顧客は製品が遅れて届いたことは忘れても製品の質の悪さを忘れることはない。」という言い伝え (dictum)
2. aptly: 適切に

2. On-time delivery(製品の配送時間)
Nothing irritates¹ a customer more than not receiving a delivery on the promised date. When a delivery is late, plans at the customer's end have to be redrawn, resource allocation has to be shifted, and all subsequent actions have to be rescheduled, causing the customer a lot of inconvenience.

1. írritate: 苛立たせる

3. Money(ベンダーに支払った額)
This refers to money the customer is paying. It is not uncommon for escalation clauses¹ to be built into contracts. When the vendor chooses to apply an escalation clause and to bill more money, it greatly inconveniences the customer. The customer must obtain sanctions and approvals for the extra payout², as well as answer quite a few questions in the process. In short, price escalations irritate customers.

1. escalátion claúse: 価格の上昇に関する条項
2. páyòut: 支払い

4. Issue factor(プロジェクト実行中にベンダーが提起した仕様関連の問題)
Most projects have issue resolution mechanisms (methods to solve problems). Some vendors, in their eagerness to always interpret the specs accurately (and in their fear that they might in fact misinterpret specs), raise more issues. When valid issues are raised, the customer is usually more than happy to resolve them. But when the issues raised are trivial, the customer becomes annoyed.

1. specs: specificátions、仕様書、明細書
2. trívial: 取るに足らない、ささいな

5. Accommodation and cooperation(要望に対する対応の柔軟性)
Few projects are ever completed without changes having been requested by the customer. When the customer requests a change, the vendor should accommodate and cooperate with the customer, and implement the change without postponing the delivery and without increasing the price.

1. accómmodate: ～に対応する、配慮する

Quality Rating(製品の品質に対する満足度)

No project is ever perfect, and most times, defects may not be detectable immediately upon delivery. If defects are detected during the warranty period, the customer is happy. However, what is important is whether the defects fall into an acceptable range. Usually, the customer's expectation is "zero defects," but all professionals on quality know that "zero defects" is rarely achieved.

1. wárranty périod: 保証期間
2. whether the defects fall into an acceptable range: 不備が許容範囲内にあるかどうか

Sometimes, customers specify what is an acceptable defect density (number of defects per number of opportunities for error); other times, the defect density is implicit. Customers select vendors based on their certifications or market reputation. But reputation alone does not lend itself for measurement. Using six-sigma philosophy, we can measure and specify the expected defects based on the "sigma level" of the vendor organization.

1. defect density: 不備の割合
2. six-sigma philósophy: ばらつきを表すσ (シグマ) を使って予測される不備の割合を6レベルに分けることにより、製品管理の質レベルを測る手法

If an organization is at 6-sigma level, then the expected defects from that organization total three defects for every million opportunities. If the organization is at 5-sigma level, the expected defects total three defects for every 100,000 opportunities. At 4-sigma level, three defects for every 10,000 opportunities. At 3-sigma level, three defects for every 1,000 opportunities.

The expected number of defects delivered should be contrasted against the actual number of defects delivered.

Defects begin to be counted during the acceptance testing stage because they can be discovered by the customer just as they can be in pilot-runs, during live or production runs, throughout the warranty period, and afterward.

1. pilot-run: 試験 (実験) 運行

Normally, defects are classified as one of three categories: critical, major, and minor. I use only the critical and major defects, since minor defects can sometimes merely be a difference in perception—the customer may perceive as a defect what the vendor may not consider a defect.

The defect density is computed as defects per unit size, or conversely, as units of product per one defect. The size is usually measured as lines of code (LOC), function points (FPs), or any other size measure used in the organization. What is important is to select one size and use it for all measurements.

1. lines of code (LOC): コードの行数 (ソフトウェアの規模の測定に使用する)
2. function point (FP): ファンクション・ポイント、機能の数 (ソフトウェアの規模の測定に使用する)

Here is the formula to compute a quality rating (QR) for customer satisfaction:

1. quality rating (QR): 品質の評点

QR(品質評点) = (actual defect density – accepted defect density) ÷ accepted defect density

1. actual defect density: 実際あった不備の割合
2. accepted defect density: 許容できる不備の最高割合

If the actual defect density is less than the accepted defect density, then this metric will be negative, meaning customer expectations have been exceeded. If the actual defect density is the same as the accepted defect density, then this metric will be zero—customer expectations have been fully met. If the actual defect density is more than the accepted defect density, then this metric will be positive, and it means customer expectations have not been fully met.

Delivery Schedule Rating(製品の配送時間に対する満足度)

Nothing is more frustrating than not receiving a delivery on an agreed-upon day. This frustration may be eased if somebody calls to tell you that the delivery is going to be delayed, but the frustration is there just the same. The funny part is, even if a delay is the result of a change that the customer requested, late delivery still frustrates the customer. It is as if the customer is thinking, "Can't they accommodate this teeny-weeny change without postponing the delivery date? Vendors always take any opportunity to delay delivery!"

1. téeny-wéeny: tinyの幼児語、ちっちゃな、ほんのちょっぴりの

Oftentimes, vendors prefer to compromise on quality than to delay delivery. The philosophy is this: it will take some time for the customer to unearth the defect, but it takes no time for the customer to come down heavily if delivery is not on time. Excuses like "Sorry for the defect; here is the corrected version" or "In our fervent efforts to deliver on time, this defect crept in" can be quite convincing.

1. ùnéarth: 明るみにする、見つけ出す
2. come down heavily: 顧客の配送元への問い合わせが殺到する
3. férvent: 熱心な、切なる
4. creep in: 知らず知らずのうちに入り込む

Customers might forget delayed deliveries, but they seldom forget poor quality. When asked for references, they normally highlight the quality a vendor provides over on-time delivery. That is the reason I place this aspect as second in importance when determining customer satisfaction.

To compute this metric, we contrast accepted delivery with actual delivery. But which date should you use as the accepted delivery date? To compute the highest rating possible, take the latest accepted delivery date. To derive a true customer satisfaction rating, then take the date that is on the purchase order. Some organizations use both—one for internal purposes and one for the external purposes.

1. purchase order: 発注書

The formula for computing a delivery rating (DR) for customer satisfaction is as follows:

DR(配送の評点) = (actual days taken for delivery – accepted days for delivery) ÷ accepted days for delivery

1. delivery rating (DR): 製品の配送評点
2. actual days taken for delivery: 配送に実際にかかった日数
3. accepted days for delivery: 配送にかけるのに許容できる日数

To determine actual days taken for delivery, use the number of calendar days between the date of the purchase order and the date on which delivery was actually effected. To determine the accepted days for delivery, use the number of calendar days between the date of the purchase order and the date of delivery specified on the purchase order.

1. actually effécted: 実際に実施される

If actual delivery was made before the accepted delivery date, then this metric will be negative, meaning customer expectations have been exceeded. If actual delivery was made on the accepted delivery date, then this metric will be zero—customer expectations have been fully met. If actual delivery was made later than the accepted delivery date, then this metric will be positive, and it means customer expectations have not been fully met.

Price Rating(ベンダーに支払った額に対する満足度)

Obviously, no vendor can bill the customer for an amount that was not agreed to by the customer—that is if the vendor expects his invoice to be respected in full and without issue. Why is this an important factor? Because sometimes contracts are drawn up using an hourly rate with a maximum amount, allowing some variance on either side. In such cases, the final billed amount can either be lower or higher than the specified amount.

1. draw up cóntract: 契約書を作成する

When a price escalation clause is implemented or an additional payment is requested against a change, some negotiating usually occurs before the customer accepts the escalation; the amount accepted might not be the same as requested by the vendor. The fact that extra money is being requested and the resultant negotiations can certainly frustrate the customer.

Whenever the customer has to pay more than the purchase order value, the customer is dissatisfied. Needless to say, the customer is certainly pleased when the vendor charges less money than the amount specified on the purchase order.

To compute the price rating (PR), use the price agreed to (before taxes) on the original purchase order and the final amount billed. Here is the formula for computing customer satisfaction in this area:

1. price rating (PR): 価格の評点

PR(価格の評点) = (actual amount billed ¹– price on the purchase order²) ÷ price on the purchase order

1. actual amount billed: 実際に請求された額
2. price on the purchase order: 発注書に記載された額

If the actual amount billed was less than the purchase order price, then this metric will be negative, meaning customer expectations have been exceeded. If the actual amount billed was equal to the purchase order price, then this metric will be zero—customer expectations have been fully met. If the actual amount billed was more than the purchase order price, then this metric will be positive, and it means customer expectations have not been fully met.

Issue Rating(ベンダーによる仕様関連問題の提起に対する満足度)

Issues crop up¹ during project execution mainly because of unclear specifications or a lack of understanding the specs. Issues may also occur because of a conflict or an error in the requirements.

1. crop up: 思いもかけず、ひょっこり)表れる

When the vendor raises an issue whose origin is attributable to the customer, the customer's satisfaction is not usually affected. However, the customer's satisfaction does become affected if the issues raised are due to the vendor's improper understanding of the requirements. Customers expect any shortfall in exhaustive¹ requirements specifications to be bridged by the vendor. Failure to meet these expectations cause dissatisfaction in customers.

1. exháustive: 包括的な、完全な

To compute an issue rating (IR)¹, use the issue density(ID)². While we can easily compute actual ID (issue density), there is no accepted measure for an acceptable ID (issue density). We also use software size for computing ID (issue density). While issues can directly relate to requirements, we cannot use the number of requirements, as the method for defining requirements can vary the number significantly.

1. íssue rating (IR): 問題提起の評点
2. issue density (ID): 問題提起の割合

Thus, the ID is computed as follows:

ID(提起された問題の割合) = number of issues raised¹ ÷ software size²

1. number of issues raised: 問題提起の数
2. software size: ソフトウェアの規模 (LOCやFPを使って測定する)

Software size can be any software size measure, such as lines of code(LOC) or function points(FP). Since there is no universally acceptable ID, an organizational standard should be defined and continuously improved.

The formula for computing IR for customer satisfaction is as follows:

IR(問題提起の評点) = (actual ID¹ – standard ID²) ÷ standard ID

1. actual ID (issue density): 実際の問題提起の割合
2. standard ID (issue density): 問題提起の標準割合

If the actual ID(issue density) was less than the standard ID(issue density), then this metric will be negative, meaning customer expectations have been exceeded. If the actual ID(issue density) was the same as the standard ID(issue density), then this metric will be zero—customer expectations have been fully met. If the actual ID(issue density) was more than the standard ID(issue density), then this metric will be positive, and it means customer expectations have not been fully met.

Cooperation Rating(ベンダーの対応の柔軟性に対する満足度)

Most projects would not be complete without a few change requests from the customer—software maintenance projects run on these. But since change requests are commonly implemented before delivery, how then do they give rise to customer dissatisfaction?

Change requests cause additional work for the vendor, and their impact is felt in two ways: revised delivery schedule and higher cost. In some cases, the vendor absorbs both, and in others, the vendor absorbs the impact on price only and passes the impact on delivery schedule on to the customer. Still in other cases, the vendor absorbs the impact on delivery schedule and passes the impact on price on to the customer. In the remaining cases, the changes are rejected.

Of course, the customer is happy when change requests are accepted without impacting the price or the delivery schedule. But since this does not always happen, that is why we compute a cooperation rating (CR), the formula of which is the following:

CR(協力性の評点) = (no. of change requests received¹ – no. of change requests implemented without affecting delivery date or price²) ÷ no. of change requests received

1. no. of change requests received: no.=number、受け取った変更依頼の数
2. no. of change requests implemented without affecting delivery date or price: 予定していた配送日や価格に影響が及ぶことなく受け入れられた変更依頼の数

If the number of change requests received were the same as the number of change requests implemented without affecting either delivery schedule or price, then this metric will be zero, meaning customer expectations have been fully met. If the number of change requests received were greater than the number of change requests implemented without affecting either delivery schedule or price, then this metric will be positive, and it means customer expectations have not been fully met.

There is no way to exceed customer expectations in this rating.

Composite Customer Satisfaction Rating(顧客企業の満足度総合評点)　

Having computed all five ratings critical to achieving customer satisfaction, we are ready to compute the composite customer satisfaction rating (CCSR)¹.

1. composite customer satisfaction rating (CCSR): 顧客企業の満足度総合評点

Obviously, all five ratings do not carry the same importance in achieving customer satisfaction. These ratings can also vary from organization to organization, and from customer to customer. Some customers may perceive quality as being the most important aspect of a product or a service, while some may perceive delivery as the most important aspect. Still for others, the highest of importance might be placed on price. Given these differences in customers' perceptions and preferences, it is necessary to assign weights to each of the five ratings in order to arrive at a reasonably accurate CCSR.

The sum of all the weights must equal 1.00 in order to calculate a meaningful CCSR. Table 1 shows an example of how weights can be distributed.

Table 2. Example of apportioned weights among the five ratings.
(5種類の評点に割り当てる比重の一例)

The formula to compute composite customer satisfaction rating(CCSR) is this:

CCSR (顧客企業の満足度総合評点) = 5 – (QR*w1 + DR*w2 + PR*w3 + IR*w4 + CR*w5)

This formula gives the composite customer satisfaction rating (CCSR) on a 5-point scale. It is possible for the composite customer satisfaction rating (CCSR) to be greater than 5 in some cases. When this happens, it means that customer expectations have been exceeded.

Use of composite customer satisfaction rating (CCSR)
(顧客企業の満足度総合評点の利用の仕方)

While I do not advocate doing away with customer satisfaction surveys(CSSs) altogether (ultimately, what the customer perceives is also important), consider these facts: Only one person in a customer organization fills out customer satisfaction surveys(CSSs), despite the fact that many people in the organization may use the product. This one person's expectations can be managed, making it possible to calculate an accurate rating. But the other users (some of whom could be decision makers) can certainly still unearth the defects in the product.

This is to say that perception-based ratings alone cannot be relied upon. Contrasting customer satisfaction survey(CSS) ratings with composite customer satisfaction rating(CCSR) allows organizations to improve their processes.

Suppose that the internal customer satisfaction surveys(CSS) is in agreement with a customer satisfaction surveys(CSS) rating. This means that the customer's perception is in sync with¹ reality, and that customer expectations are being managed as they should be. The organization's strengths are equal in service and expectation management, giving a realistic picture to management. In this case, the organization needs to take corrective action² based on the rating should it be poor.

1. in sync with ～: sync = synchronizátion 、～と同調して
2. corréctive action: 修正措置、是正処置

Suppose that the internal composite customer satisfaction rating(CCSR) is way below the CSS rating. This means that the customer's perception of an organization's service is better than the service is in reality. This is not of any benefit to the organization, because if it continues to praise itself based on the customer's perception that its level of service is high, then the organization will head toward decay. Resources will continue to place emphasis on expectation management rather than on service, thus never improving services. In this case, resources need to be trained in order to improve service.

Now suppose that the internal composite customer satisfaction rating(CCSR) is way above the customer satisfaction surveys(CSS) rating. In such a case, the customer's perception of an organization's service is poorer than the service is in reality. This shows that the organization is concentrating on service without any concern for expectation management. Interpersonal relations and communication with the customer are being neglected. Here, resources need to be trained in expectation management.

There is scope¹ in the composite customer satisfaction rating(CCSR) method for organization-based adaptation. Some of the five ratings may be dropped or substituted, or new ones may be added to suit the specific organization.

1. scope: 余地。顧客満足度の総合評点 (CCSR) の計算方式を各企業に合わせて調整できるということ。

著者について

Murali Chemuturi氏
Indian Institution of Industrial Engineeringの特別研究員 (フェロー) であるとともにComputer Society of Indiaのシニア会員。現在、ソフトウェア開発業界のベテランとしてChemuturi Consultants社を引率し、ソフトウェア開発プロセスの質の向上、研修などのサービスを提供している。Chemuturi氏への連絡先はmurali@chemuturi.comまで。