Attribute Agreement Analysis Gage r&r

• Every time someone makes a decision – such as, “Is this the right candidate?” – it is critical that the decision-maker would select the same choice again and that others would reach the same conclusion. Attribute agreement analysis measures whether or not several people making a judgment or assessment of the same item would have a high level of agreement among themselves.
• Benefits

• Helps to characterize the quality of the data
• Determines the area of non-agreement
• Helps in calibrating appraisers, judges, or assessors for a higher level of agreement
• Easy to analyze with statistical software or a specialized worksheet
• How to Use

• Step 1.  Set-up a structured study where a number of items will be assessed more than once by more than one assessor. Have the items judged by an expert, which will be referred to as the “standard” (can be one person or a panel – see table below).
• Step 2.  Conduct the assessment with the assessors in a blind environment. They do not know when they are evaluating the same items, and they do not know what the other assessors are doing.
• Step 3.  Enter the data in a statistical software package or an Excel spreadsheet already set up to analyze this type of data (built-in formula).
• Step 4.  Analyze the results: Is there a good agreement between appraisers? Each appraiser vs. the standard? All appraisers vs. the standard?
• Step 5.  Draw your conclusions and decide on the course of action needed if the level of agreement is below a set threshold. Usually, > 80 percent is considered a good level of agreement.

Gage R&R

• The Gauge R&R method analyzes how much of the variability in your measurement system is due to operator variation (reproducibility) and measurement variation (repeatability). Gauge R&R studies are available for many combinations of crossed and nested models, regardless of whether the model is balanced.
  • Before performing a Gauge R&R study, you collect a random sample of parts over the entire range of part sizes from your process. Select several operators at random to measure each part several times. The variation is then attributed to the following sources:
  • The process variation, from one part to another. This is the ultimate variation that you want to be studying if your measurements are reliable.
  • The variability inherent in making multiple measurements, that is, repeatability.
  • A Gauge R&R analysis then reports the variation in terms of repeatability and reproducibility
  • The variability is due to having different operators measure parts—that is, reproducibility.