PROJECTmagazine

Home

Bid-Evaluation & Procurement

The Pairwise comparison [AHP] technique is better than the Matrix method

When purchasing products or contracting services, we evaluate bids. This critical decision-making activity has two global requirements: Objectivity and Transparency.

Subjectivity-(the opposite of objectivity) is a word we use to define our values, emotions, experiences, ego, etc. As long as human beings make decisions, there is no way to eliminate subjectivity from the process. Studies have shown subjectivity to be the driving force in decision-making, and all our decisions will reflect our subjectivity. Unfortunately, excessive subjectivity tends to bias our decisions, and prevents us from reaching the optimum decision. Thus it is necessary to temper this subjective influence.

Transparency-the second requirement of today's decision-making needs is defined as-clarity of a process that conforms to established audit principles. If the final decision is not transparent, non-winning candidates or special interest groups might accuse the decision-makers of bias, and in some cases they might even initiate legal proceedings. This can result in unwarranted delays and even significant cost overruns. Indeed, even if the selection process is subject to strict monitoring, proving this to management, the public and to monitoring authorities can be a daunting task.

Held Hostage by the Matrix Method

The Matrix method (sometimes referred to as the Expected Value method) has been the only option available to decision-makers to evaluate bids; this is woefully inadequate. This method requires one to compare all the bidders, as a group, for one criterion at a time, and assign points to them.

We propose the XpertUS Decision Support System [XDSS], which uses the AHP pairwise comparison technique to rank candidates. In this technique, one would compare candidates (bidders) in pairs, as before, but considering one criterion at a time. The AHP technique was originally developed by Dr. Saaty [Univ. of Pittsburgh] and has been used extensively in the decision-making arena.

However, just using the AHP technique, without a thorough understanding of the fundamentals of decision-making is akin to trying to use a calculator, without a true understanding of the problem. XDSS provides a holistic approach to decision-making.

Criteria Segregation

All decisions are governed by a set of criteria - these are the factors that impact your decision. The

1. Prerequisites (for candidate selection)

2. Obligatory Criteria (evaluation criteria)

3. Desirables (nice-to-have features).

criteria you identify fall into two groups; additionally, there are minimum requirements (prerequisites) that need to be satisfied (see box). Specific guidelines will determine when a bidder must be eliminated. If these In/Out guidelines are not well defined, you might eliminate a perfectly acceptable candidate from being considered. You do not require an elaborate method for this process-of-elimination; a simple table would suffice. These prerequisites manifest themselves in two forms. "A Priori" and "Limiting Conditions".

Criteria associated with "a priori" prerequisites should not be included in the ranking process; they serve only to select candidates. However, criteria associated with "limiting conditions", while playing a vital role in candidate selection, remain part of the evaluation. Even experienced evaluators tend to confuse prerequisites (which must be satisfied as a minimum) with obligatory criteria (which are satisfied to various degrees); the latter are the only ones used in the bid-evaluation (candidate ranking process). In addition to the obligatory criteria there are desirable criteria, which are nice-to-have features.

If you have too many criteria - use the pairwise comparison technique to whittle down these criteria to a reasonable number of about 12. The rejected criteria are the desirables, hence should not be considered in the evaluation unless the final ranking does not present a clear choice.

Assigning Criteria Weights

Presently, evaluators use one of two common methods of assigning

1. The distribution technique-where 100% of the weights is distributed among all the criteria.

2. The scaling technique-where each criterion is assigned a number, on a scale of 1 -10 (1 = low preference) and normalized (add all the points and divide each number by the total and multiply by 100). You may use any range.

criteria weights (see box). Sadly, both these methods are victims of a high degree of subjectivity. The most reliable way to assign criteria weights is to use the pairwise technique. This technique consist of comparing criteria in pairs, and indicating your preference or the superiority, of one over the other on a scale of-equal, weak, moderate, strong and absolute. Then you can use an algorithmic technique such as XpertUS to determine weights (available at http://www.xpertus.com/ ).

Pairwise Comparison Is Consistent and Reliable

The method of comparing criteria and candidates determines the final outcome of an evaluation.

Consider two bidders; if you were asked to distribute 100 points among these two to reflect their quality of work (a criterion); you would do this with ease and a high degree of confidence. Let us add two more bidders; now you would have some difficulty-but still a manageable task. Now suppose you are faced with having to distribute 100 points among seven bidders, would you have the same level confidence in your final ranking-as you did with the two bidders? Probably not! You can take the skill-test available at http://www.xpertus.com/ to measure your decision-making skills.

Here we considered only one criterion; now add more criteria to your selection process. This is when subjectivity and human limitations begin to adversely influence your decisions. By recognizing that you could decide with ease when faced with two options, you have confirmed that a method, which uses a pairwise comparison technique, is superior to one that relies on group comparison.

Note that the Pairwise technique is not the same as the process-of-elimination technique used by opticians.

Sensitivity Analysis

In any evaluation, it is imperative that we study the sensitivity of our decision to various criteria. The XpertUS technique can be used to conduct such a sensitivity analysis.

Different Consolidation Techniques Add Further Confusion

Frequently, a team performs a bid evaluation. The team consists of representative (specialist) for each of the evaluation criteria. Where necessary, each specialist would in turn form his/her own small group. Thus, the groups represent areas of specialized know-how.

Consolidating multiple decisions is crucial; do not underestimate its value. How one consolidates an evaluation to arrive at the final ranking depends on the manner in which the evaluation is conducted. There are two basic techniques. We recommend that you use either the Criteria-Consolidation technique or the Decision-consolidation technique, Offered by the XpertUS technique, depending on your particular problem.

For more information on how to get a free copy of the XpertUS software, visit http://www.xpertus.com/

About the Author

Dr. Wirasinghe started life as a trainee mechanic in an Engineering workshop, and went on to graduate as Valedictorian from Teesside University, U.K. specializing in Mechanical Engineering. Subsequently he obtained his Ph.D. in Fluid Mechanics, having won a scholarship from the "Confederation of British Industries".
He has published papers on topics relating to Management, Engineering and Economics and has carved out a niche as a "decision-maker" and a "creative-thinker", to shatter the myth and math of decision-making.
Initially, this technique was used for deciding where to drill for oil, based on seismic studies from different locations. In 1990, Dr. Wirasinghe adopted it for general applications. Today, his process is used in disciplines as diverse as Budgeting, Career Planning, Safety, Purchasing, Bid Evaluation, Recruiting, Performance Evaluations, Customer Rankings, etc.