SC QUALITY GUILDGauge selection and resolution checker
SCQUALITYGUILD.COM/TOOLS/GAUGE-RESOLUTION-10-TO-1.HTML
Check whether each instrument can discriminate and is accurate enough for the tolerance it will judge, converting between inches and millimeters exactly.
ON THE EXAM · CQI II.C.1, II.C.2, I.E, CQT IV.B.2, IV.B.5 · FREE · NOTHING YOU TYPE LEAVES YOUR BROWSER
Q-BotClick for an explanation of this tool.For educational use. This page is a study aid. The method behind it has far more depth and detail than one page and one tool can cover, and what you see here is not everything the method offers or everything this site offers. For full understanding and application, use the governing standards and texts, and seek further education, training or qualified advice. Results depend on the data you enter, so check them before relying on them.
The rule of ten
The rule of ten, also called the 10:1 rule, says a measuring instrument should resolve to at least one tenth of the tolerance it is used to judge. A characteristic toleranced 4.000 to 4.050 mm (0.050 mm total) needs an instrument that reads to 0.005 mm or finer. Below that ratio, the readings fall into so few steps across the tolerance that parts near the limits cannot be sorted reliably.
Resolution is not accuracy
- Resolution (discrimination): the smallest change the instrument displays.
- Accuracy: how close the reading is to the true value, stated as ± on the calibration certificate or data sheet.
- Precision: how closely repeated readings agree with each other.
A digital caliper that displays 0.0005 in but is accurate only to ± 0.001 in displays more digits than it can support. The tool therefore runs a second check: the ± tolerance divided by the ± accuracy. A common minimum is 4:1, the same ratio often used as a test accuracy ratio in calibration. Your quality manual or customer may set other ratios; enter them in the first section.
Inch and millimeter conversion
One inch is exactly 25.4 millimeters by definition, so conversions carry no rounding error of their own. The error comes from rounding too early. Carry at least one more decimal place than the tolerance, convert, and round only the final result. A 0.0005 in resolution is 0.0127 mm; rounded to 0.01 mm it would look better than it is.
Reading the result
Each row shows the limits in the other unit, the tolerance-to-resolution ratio and the tolerance-to-accuracy ratio. When an instrument fails, the checks state the finest resolution needed. The usual fix is the next class of instrument: caliper to micrometer, rule to caliper, or a dedicated bore or indicator gauge.
On the CQI exam
Expect questions that give a tolerance and several instruments and ask which is appropriate, or that mix inch and metric values. Use the total tolerance, convert before comparing, and do not confuse resolution with accuracy.