search for   

 

Six Sigma Analysis of Vitamin D Measurement Using External Quality Assessment Program
Korean J Clin Lab Sci 2020;52:91-97  
Published on June 30, 2020
Copyright © 2020 Korean Society for Clinical Laboratory Science.

Myungsuk Ji

Department of Laboratory Medicine, Kangbuk Samsung Hospital, Seoul, Korea
Correspondence to: Myungsuk Ji
Department of Laboratory Medicine, Kangbuk Samsung Hospital, 29 Saemunan-ro, Jongno-gu, Seoul 03181, Korea
E-mail: ms.gee@samsung.com
ORCID: https://orcid.org/0000-0001-7536-0367
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract
Standardization of vitamin analysis continues around the world, and much effort has been made to improve the accuracy of the results. This study analyzed the sigma metrics of the vitamin D test using the external quality assessment (EQA) program. Sigma metrics is used for quantitative tests performed in the laboratory, and the test results can be objectively visualized in terms of quality. This analysis was performed based on the accuracy of the College of American Pathologists (CAP) using the results of the 2019 accuracy-based vitamin D (ABVD) survey, and about 300 laboratories participated in the survey. Reference values were obtained by the Center for Disease Control and Prevention (CDC) reference laboratory. At six different concentrations, the sigma metrics were analyzed to be 1.00, 1.85, 2.42, 1.01, 1.54 and 0.78, respectively. An average of 1.43 sigma metrics was determined. In particular, only positive biases for ABVD-16 and 17 were shown in the liquid chromatography tandem-mass spectrometry (LC-MS/MS), which is the standard method for vitamin D determination when compared to the reference values. The causes of the difference can be explained by cross reactivity to various vitamin D metabolites. Laboratories need to improve their overall performance.
Keywords : Liquid chromatography-tandem mass spectrometry, Sigma metrics, Vitamin D
INTRODUCTION

Vitamin D plays an important role in maintaining calcium and phosphate homeostasis, essential for bone health [1]. Vitamin D increases calcium absorption in the intestine and reduces calcium excretion in the kidneys [2]. 25-OH vitamin D (25-OH D) is the main circulating form of vitamin D and reflects the state of it [3, 4]. Vitamin D deficiency is associated with increased risk of several chronic diseases, including high blood pressure, cardiovascular disease, some cancers, diabetes, human immune diseases, and infectious diseases [5-10]. Vitamin D metabolites are fat-soluble, strongly bound to vitamin D-binding proteins, and have very low blood levels, making them difficult to measure [11]. Methods of measuring 25-OH D include immunoassays, competitive protein binding assays, radioimmunoassay (RIA), high performance liquid chromatography (HPLC), and liquid chromatography tandem-mass spectrometry (LC-MS/MS). Automated analysis systems were developed by several manufacturers. One of the biggest problems with the current 25-OH D test is that it is difficult to interpret and standardize the results due to significant differences between the test methods and the laboratory [12-15]. To assess the 25-OH D test results and accuracy, Laboratories must regularly participate in external quality assessments. CAP conducts an accuracy-based survey that evaluates external quality assessments based on actual values measured in a reference laboratory. Laboratories should use calibrated materials with proven traceability to improve the accuracy of their 25-OH D assay. In recent years, six sigma has been used to evaluate analytical performance in laboratories [16, 17]. Sigma metrics is calculated using coefficient of variation (CV), bias, and total allowable error (TEa). Six Sigma analysis enables laboratories to measure current processes to give the laboratory some measure of performance and to obtain objective indicators for the desired quality improvement. The purpose of this study is to analyze six sigma metrics using external quality assessment of vitamin D.

MATERIALS AND METHODS

1. Vitamin D proficiency test survey data

It used the ABVD proficiency test (PT) data from CAP in 2019. This PT was conducted twice a year and three fresh serum specimens were used for each test. 307 laboratories were participated in the 2019 ABVD PT survey, and major test instruments and methods were investigated.

2. Vitamin D reference value measurement

The reference target value for 25-OH D in each specimen was determined by the isotope-dilution LC-MS/MS reference method procedure performed at the Fat-soluble Nutrients Laboratory, Nutritional Biomarkers Branch at the Centers for Disease Control and Prevention (Atlanta, Georgia). The external quality assessment (EQA) specimens were comprised of reference target value: 9.45, 25.10, 34.60, 22.29, 36.70 and 8.75 ng/mL. The reference laboratory also participates in a vitamin D standardization program coordinated by the National Institutes of Health (NIH).

3. Vitamin D allowable criteria

TEa represents an acceptable difference from the reference target value. The 25-OH D TEa requires a value within 25% of the CDC reference value or less than 5 ng/mL [18]. In this study, Many laboratories using LC-MS/MS have not reported values within acceptable limits, which are likely to be caused by calibration problems, variable sensitivity of measurement procedures at a low concentration (<0.6 ng/mL) of 25-OH D2, and the detection of a C3-epimer at 25-OH D3.

4. Statistical analysis

The quantitative data provided in the participant summary include the mean, median, standard deviation (SD), CV, TEa, the lowest and highest values reported for each peer group, statistics are for methods with a peer group N≥10. The vitamin D was assay performed on four instruments. They were ADVIA Centaur, Centaur XP/XPT (Siemens, Munich, Germany), LC-MS/MS, Architect i System (Abbott Diagnostics, Lake Forest, IL, USA) and LIAISON (DiaSorin, Saluggia, Italy). TEa values for various parameters are obtained from CAP. Bias is a systematic difference between results obtained with laboratory test methods and those obtained with an accepted reference method. The bias was calculated from external quality assessment values using the following formula: %Bias = (Laboratory mean ‒ Group mean) × 100 / Group mean. CV is the analytical coefficient of variation of the test method. %CV = (Standard deviation / Laboratory mean) × 100%. The Sigma metric was calculated with the following formula: Sigma = (%TEa ‒ %Bias) / %CV [19]. All data was calculated in Microsoft Excel 2010 (Microsoft, Redmond, Washington). The analytical performance of 25-OH D assay was evaluated according to the obtained sigma levels. The normalized method determination chart was used to describe the sigma matrix.

RESULTS

1. 2019 Vitamin D sigma metrics

Using the EQA reference target value of CAP, the distribution of sigma value for the 25-OH D assays was as follows Table 1, applying the TEa of 25-OH D and the criteria for evaluating the analytes used in each survey. At six concentrations the sigma metrics were 1.00, 1.85, 2.42, 1.01, 1.54 and 0.78, respectively.

Sigma matrix of vitamin D analysis using six reference target values

ABVD-13 ABVD-14 ABVD-15 ABVD-16 ABVD-17 ABVD-18
Reference (ng/mL) 9.45 25.10 34.60 22.29 36.70 8.75
Sigma metrics 1.00 1.85 2.42 1.01 1.54 0.78

Abbreviation: ABVD, accuracy-based vitamin D.



2. 2019 Vitamin D sigma metrics by instruments

The sigma metrics calculated for each instrument are shown Table 2. The best sigma metrics was that Architect i System averaged 2.80 sigma, followed by LC-MS/MS (2.09 sigma), Liaison (1.95 sigma), and ADVIA Centaur, Centaur XP/XPT (1.09 sigma).

Sigma metrics of vitamin D assay conducted by instruments

ABVD-13 ABVD-14 ABVD-15 ABVD-16 ABVD-17 ABVD-18
Reference (ng/mL) 9.45 25.10 34.60 22.29 36.70 8.75
ADVIA Centaur 0.92 0.79 1.46 1.00 1.54 0.80
LC-MS/MS 1.59 2.35 3.02 1.83 2.08 1.67
Architect i System 3.91 3.46 3.72 1.78 1.40 2.52
LIAISON 1.92 2.70 2.63 1.24 1.63 1.59


3. 2019 Vitamin D normalized sigma metric method decision chart

Normalized charts express bias and imprecision as a percentage of TEa for analyte. Each point represents the %bias (y-axis) and the %CV (x-axis) for one of six concentrations. Sigma metric values are shown from Figure 1 to Figure 5.

Fig. 1. Normalized sigma metric method decision chart for 2019 Vitamin D.
Fig. 2. Normalized sigma metric method decision chart with ADVIA Centaur, Centaur XP/XPT.
Fig. 3. Normalized sigma metric method decision chart with LC-MS/MS.
Fig. 4. Normalized sigma metric method decision chart with Architect i System.
Fig. 5. Normalized sigma metric method decision chart with LIAISON.

4. ABVD-16 and 17

Figure 6 and 7 showed positive bias only for LC-MS/MS method. In ABVD-16 and 17 specimens, the 25-OH D2 values were measured at 5.59 and 11.6, respectively.

Fig. 6. ABVD-16 bias plot for instruments.
Fig. 7. ABVD-17 bias plot for instruments.
DISCUSSION

In the clinical laboratory, it is relatively easy to participate in accuracy-based surveys to verify the accuracy of test methods. Unlike the way external quality assessment results are interpreted as the average of the participating groups, accuracy-based PT surveys evaluate the results with actual values measured by standard methods. The use of sigma metrics for quantitative test items performed in the laboratory allows you to objectively visualize test results in terms of the quality of the test results. Six Sigma can help you achieve the desired quality in laboratory testing processes and measurements [20]. Six sigma improves process quality by analyzing and eliminating error sources to reduce the variability of how they are currently used. A low sigma value (<3 sigma) in the clinical laboratory identifies the test, indicating that action must be taken to improve the quality of the analysis and that the laboratory should use alternative methods. In a CAP survey, the performance of five test subgroups was evaluated in 307 laboratories. Study has shown that the analyzer used for routine 25-OH D measurements in the laboratory is lower than the suggested minimum 3 sigma. The limitation of this study was that it was evaluated using only the mean and CV of the same group. The reason that Architect i System was able to give the best sigma metrics is because CV value, a factor when calculating sigma metrics, showed better result than other test methods. Results near the reference value were best with LC-MS/MS analysis. Nevertheless, the reason for the low sigma metric is that the CV value was relatively large. There are 3 possible explanations for this. In house calibrators, limit of detection (LOD) quantification 25-OH D2, samples with low 25-OH D and a high percentage of 25-OH D2 [21]. The measurement of 25-OH D is mainly based on immune analysis. In many cases, however, there were disadvantages in distinguishing between 25-OH D3 and 25-OH D2, and measurements of 25-OH D have been reported to cross-react with 24, 25-dihydroxyvitamin D3 [22]. Measurement of vitamin D metabolites by LC-MS/MS has been the subject of many studies since 2005 [23-29]. The reference method of the vitamin D assay was LC-MS/MS, in which 25-OH D2, 25-OH D3 and D3 epimers can be measured separately. 25-OH D2 can only be obtained through dietary supplement intake, and there are some claims that the biological activities of 25-OH D3 and 25-OH D2 are different [30, 31]. It is important to distinguish 25-OH D2. In the United States, in all vitamin D assays used for clinical diagnosis, the total 25-OH D must have the ability to detect the sum of the 25-OH D2 and 25-OH D3 [32]. Especially in this study, the ABVD-16 and 17 specimens showed positive biases only for the LC-MS/MS method in Figure 6 and Figure 7. In these two specimens, the 25-OH D2 values were measured at 5.59 and 11.6, and the remaining specimens at <0.6. Because vitamin D reports its total value, a reflected in the results, 25-OH D2 cannot be separated by any methods other than LC-MS/MS. Several studies have reported negative bias results for automated assays compared to LC-MS/MS method for measuring 25-OH D [33, 34]. Discrepancies between vitamin D total immunoassays are constantly raised and reported in the several studies. [35, 36]. Manufacturers using reference values and the most inaccurate methods are encouraged to improve performance, especially those that consistently represent large bias and CVs, which are need to be observed carefully. It need to improve your overall performance. Six Sigma metrics can be used as a self-assessment method in clinical laboratories to formulate quality strategies. In order to get accurate test results, it is very helpful to implement these metrics into the analytical process in the laboratory.

요 약

비타민 D 검사의 표준화는 전 세계적으로 계속되고 있으며, 결과의 정확성을 향상시키기 위해 많은 노력을 기울였다. 본 연구의 목적은 외부 품질 평가(EQA) 프로그램을 이용하여 비타민 D 검사의 시그마 메트릭스 값을 분석하는 것이다. 시그마 메트릭스는 검사기관에서 수행한 정량적 시험에 사용되며, 분석결과는 품질 측면에서 객관적으로 시각화할 수 있다. 이 분석은 약 300개의 검사기관이 참여한 가운데, CAP의 외부정도관리 프로그램인 2019년 정확도 기반 비타민 D (ABVD) 조사 결과를 이용하여 수행하였다. CDC 표준검사기관에서 얻어진 표준 값 6개의 서로 다른 농도에서는 시그마 메트릭 값이 각각 1.00, 1.85, 2.42, 1.01, 1.54 및 0.78로 분석되었다. 평균 1.43 시그마 메트릭 값을 보여주었다. 특히, 검체 ABVD-16, 17의 결과값에서의 양성 편향은 비타민 D 측정을 위한 표준검사법인 액체 크로마토그래피 탠덤 질량 분석(LC-MS/MS)에 대해서만 나타났다. 차이의 원인은 다양한 비타민 D 대사산물에 대한 다양한 교차 반응으로 설명된다. 검사 기관은 전반적인 성능을 개선해야 한다.

References
  1. Audran M, Kumar R. The physiology and pathophysiology of vitamin D. Mayo Clin Proc. 1985;60:851-866. http://doi.org/10.1016/s0025-6196(12)64791-0.
    Pubmed CrossRef
  2. Norman AW. From vitamin D to hormone D: fundamentals of the vitamin D endocrine system essential for good health. Am J Clin Nutr. 2008;88:491-499. http://doi.org/10.1093/ajcn/88.2.491S.
    Pubmed CrossRef
  3. Holick MF. Vitamin D status: measurement, interpretation and clinical application. Ann Epidemiol. 2009;19:73-78. http://doi.org/10.1016/j.annepidem.2007.12.001.
    Pubmed KoreaMed CrossRef
  4. Dawson-Hughes B, Heaney RP, Holick MF, Lips P, Meunier PJ, Vieth R. Estimates of optimal vitamin D status. Osteoporos Int. 2005;16:713-716. http://doi.org/10.1007/s00198-005-1867-7.
    Pubmed CrossRef
  5. Lavie CJ, Dinicolantonio JJ, Milani RV, O'Keefe JH. Vitamin D and cardiovascular health. Circulation. 2013;128:2404-2406. http://doi.org/10.1161/CIRCULATIONAHA.113.002902.
    Pubmed CrossRef
  6. Holick MF. Sunlight and vitamin D for bone health and prevention of autoimmune diseases, cancers, and cardiovascular disease. Am J Clin Nutr. 2004;80:1678s-1688s. http://doi.org/10.1093/ajcn/80.6.1678S.
    Pubmed CrossRef
  7. Mathieu C, Badenhoop K. Vitamin D and type 1 diabetes mellitus: state of the art. Trends Endocrinol Metab. 2005;16:261-266. http://doi.org/10.1016/j.tem.2005.06.004.
    Pubmed CrossRef
  8. Dawson-Hughes B, Mithal A, Bonjour JP, Boonen S, Burckhardt P, Fuleihan GEFuleihan GE, et al. IOF position statement: vitamin D recommendations for older adults. Osteoporos Int. 2010;21:1151-1154. http://doi.org/10.1007/s00198-010-1285-3.
    Pubmed CrossRef
  9. Holick MF, Binkley NC, Bischoff-Ferrari HA, Gordon CM, Hanley DA, Heaney RPHeaney RP, et al. Evaluation, treatment, and prevention of vitamin D deficiency: an Endocrine Society clinical practice guideline. J Clin Endocrinol Metab. 2011;96:1911-1930. http://doi.org/10.1210/jc.2011-0385.
    Pubmed CrossRef
  10. Woloszynska-Read A, Johnson CS, Trump DL. Vitamin D and cancer: clinical aspects. Best Pract Res Clin Endocrinol Metab. 2011;25:605-615. http://doi.org/10.1016/j.beem.2011.06.006.
    Pubmed KoreaMed CrossRef
  11. Shimada K, Mitamura K, Higashi T. Gas chromatography and high performance liquid chromatography of natural steroids. J Chromatogr A. 2001;935:141-172. http://doi.org/10.1016/s0021-9673(01)00943-8.
    CrossRef
  12. Lee J, Vasikaran S. Current recommendations for laboratory testing and use of bone turnover markers in management of osteoporosis. Ann Lab Med. 2012;32:105-112. http://doi.org/10.3343/alm.2012.32.2.105.
    Pubmed KoreaMed CrossRef
  13. Binkley N, Krueger D, Cowgill CS, Plum L, Lake E, Hansen KEHansen KE, et al. Assay variation confounds the diagnosis of hypovitaminosis D: a call for standardization. J Clin Endocrinol Metab. 2004;89:3152-3157. http://doi.org/10.1210/jc.2003-031979.
    Pubmed CrossRef
  14. Roth HJ, Schmidt-Gayk H, Weber H, Niederau C. Accuracy and clinical implications of seven 25-hydroxyvitamin D methods compared with liquid chromatography-tandem mass spectrometry as a reference. Ann Clin Biochem. 2008;45:153-159. http://doi.org/10.1258/acb.2007.007091.
    Pubmed CrossRef
  15. Glendenning P, Taranto M, Noble JM, Musk AA, Hammond C, Goldswain PRGoldswain PR, et al. Current assays overestimate 25-hydroxyvitamin D3 and underestimate 25-hydroxyvitamin D2 compared with HPLC: need for assay-specific decision limits and metabolite-specific assays. Ann Clin Biochem. 2006;43:23-30. http://doi.org/10.1258/000456306775141650.
    Pubmed CrossRef
  16. Westgard JO, Westgard SA. The quality of laboratory testing today: an assessment of sigma metrics for analytic quality using performance data from proficiency testing surveys and the CLIA criteria for acceptable performance. Am J Clin Pathol. 2006;125:343-354.
    CrossRef
  17. Milinković N, Ignjatović S, Šumarac Z, Majkić-Singh N. Uncertainty of measurement in laboratory medicine. J Med Biochem. 2018;37:279-288. http://doi.org/10.2478/jomb-2018-0002.
    Pubmed KoreaMed CrossRef
  18. Chemistry Resource Committee. ABVD Participant Summary [Internet]. Northfield, IL: College of American Pathologists; 2013 [cited by 2020 Feb 19].
    Available from: https://www.cap.org/member-resources/councils-committees/accuracy-based-testing-committee-participant-summary-report-discussions.
  19. Westgard JO, editor. Basic method validation. 2nd ed. Madison: Westgard QC Inc.; 2003.
  20. Nar R, Emekli DI. The evaluation of analytical performance of immunoassay tests by using six-sigma method. J Med Biochem. 2017;36:301-308. http://doi.org/10.1515/jomb-2017-0026.
    Pubmed KoreaMed CrossRef
  21. Erdman P, Palmer-Toy DE, Horowitz G, Hoofnagle A. Accuracy-based vitamin D survey: six years of quality improvement guided by proficiency testing. Arch Pathol Lab Med. 2019;143:1531-1538. https://doi.org/10.5858/arpa.2018-0625-CP.
    Pubmed CrossRef
  22. Lind C, Chen J, Byrjalsen I. Enzyme immunoassay for measuring 25-hydroxyvitamin D3 in serum. Clin Chem. 1997;43:943-949.
    Pubmed CrossRef
  23. Schleicher RL, Encisco SE, Chaudhary-Webb M, Paliakov E, McCoy LF, Pfeiffer CM. Isotope dilution ultra performance liquid chromatography-tandem mass spectrometry method for simultaneous measurement of 25-hydroxyvitamin D2, 25-hydroxyvitamin D3 and 3-epi-25-hydroxyvitamin D3 in human serum. Clin Chim Acta. 2011;412:1594-1599. http://doi.org/10.1016/j.cca.2011.05.010.
    Pubmed CrossRef
  24. Chen H, McCoy LF, Schleicher RL, Pfeiffer CM. Measurement of 25-hydroxyvitamin D3 (25OHD3) and 25-hydroxyvitamin D2 (25OHD2) in human serum using liquid chromatography-tandem mass spectrometry and its comparison to a radioimmunoassay method. Clin Chim Acta. 2008;391:6-12. http://doi.org/10.1016/j.cca.2008.01.017.
    Pubmed CrossRef
  25. Maunsell Z, Wright DJ, Rainbow SJ. Routine isotope-dilution liquid chromatography-tandem mass spectrometry assay for simultaneous measurement of the 25-hydroxy metabolites of vitamins D2 and D3. Clin Chem. 2005;51:1683-1690. http://doi.org/10.1373/clinchem.2005.052936.
    Pubmed CrossRef
  26. Kushnir MM, Ray JA, Rockwood AL, Roberts WL, La'ulu SL, Whittington JEWhittington JE, et al. Rapid analysis of 25-hydroxyvitamin D(2) and D(3) by liquid chromatography-tandem mass spectrometry and association of vitamin D and parathyroid hormone concentrations in healthy adults. Am J Clin Pathol. 2010;134:148-156. http://doi.org/10.1309/AJCPPIA7DFBT4GKS.
    Pubmed CrossRef
  27. Saenger AK, Laha TJ, Bremner DE, Sadrzadeh SM. Quantification of serum 25-hydroxyvitamin D(2) and D(3) using HPLC-tandem mass spectrometry and examination of reference intervals for diagnosis of vitamin D deficiency. Am J Clin Pathol. 2006;125:914-920. http://doi.org/10.1309/J32U-F7GT-QPWN-25AP.
    Pubmed CrossRef
  28. Vogeser M. Quantification of circulating 25-hydroxyvitamin D by liquid chromatography-tandem mass spectrometry. J Steroid Biochem Mol Biol. 2010;121:565-573. http://doi.org/10.1016/j.jsbmb.2010.02.025.
    Pubmed CrossRef
  29. Højskov CS, Heickendorff L, Møller HJ. High-throughput liquid-liquid extraction and LCMSMS assay for determination of circulating 25(OH) vitamin D3 and D2 in the routine clinical laboratory. Clin Chim Acta. 2010;411:114-116. http://doi.org/10.1016/j.cca.2009.10.010.
    Pubmed CrossRef
  30. Armas LA, Hollis BW, Heaney RP. Vitamin D2 is much less effective than vitamin D3 in humans. J Clin Endocrinol Metab. 2004;89:5387-5391. http://doi.org/10.1210/jc.2004-0360.
    Pubmed CrossRef
  31. Houghton LA, Vieth R. The case against ergocalciferol (vitamin D2) as a vitamin supplement. Am J Clin Nutr. 2006;84:694-697. http://doi.org/10.1093/ajcn/84.4.694.
    Pubmed CrossRef
  32. Hollis B. Assessment and interpretation of circulating 25-hydroxyvitamin D and 1,25-dihydroxyvitamin D in the clinical environment. Endocrinol Metab Clin North Am. 2010;39:271-287. http://doi.org/10.1016/j.ecl.2010.02.012.
    Pubmed KoreaMed CrossRef
  33. Saleh L, Mueller D, von Eckardstein A. Analytical and clinical performance of the new Fujirebio 25‐OH vitamin D assay, a comparison with liquid chromatography‐tandem mass spectrometry (LC‐MS/MS) and three other automated assays. Clin Chem Lab Med CCLM FESCC. 2016;54:617-625. https://doi.org/10.1515/cclm-2015-0427.
    Pubmed CrossRef
  34. Wyness SP, Straseski JA. Performance characteristics of six automated 25‐hydroxyvitamin D assays: Mind your 3s and 2s. Clin Biochem. 2015;48:1089-1096. https://doi.org/10.1016/j.clinbiochem.2015.08.005.
    Pubmed CrossRef
  35. Farrell CJ, Martin S, McWhinney B, Straub I, Williams P, Herrmann M. State-of-the-art vitamin D assays: a comparison of automated immunoassays with liquid chromatography-tandem mass spectrometry methods. Clin Chem. 2012;58:531-542. https://doi.org/10.1373/clinchem.2011.172155.
    Pubmed CrossRef
  36. Janssen MJ, Wielders JP, Bekker CC, Boesten LS, Buijs MM, Heijboer ACHeijboer AC, et al. Multicenter comparison study of current methods to measure 25-hydroxy-vitamin D in serum. Steroids. 2012;77:1366-1372. https://doi.org/10.1016/j.steroids.2012.07.013.
    Pubmed CrossRef


Full Text(PDF) Free

Cited By Articles
  • CrossRef (0)

Author ORCID Information