Воспроизводимость цитологических диагнозов при оценке жидкостных мазков шейки матки, а также иммуноцитохимической коэкспрессии р16/Ki-67 ручным и автоматическим методом
Воспроизводимость цитологических диагнозов при оценке жидкостных мазков шейки матки, а также иммуноцитохимической коэкспрессии р16/Ki-67 ручным и автоматическим методом
Tregubova AV, Tevrukova NS, Ezhova LS, Shamarakova MV, Badlaeva AS, Dobrovolskaya DA, Bayramova GR, Nazarova NM, Shilyaev AYu, Asaturova AV. Reproducibility of cytological diagnoses in evaluating liquid cervical smears and immunocytochemical co-expression of p16/Ki-67 using manual and automatic methods. Gynecology. 2022;24(6):499–505. DOI: 10.26442/20795696.2022.6.202009
Воспроизводимость цитологических диагнозов при оценке жидкостных мазков шейки матки, а также иммуноцитохимической коэкспрессии р16/Ki-67 ручным и автоматическим методом
Цель. Оценить воспроизводимость цитологических диагнозов при оценке жидкостных мазков шейки матки, а также иммуноцитохимической коэкспрессии р16/Ki67 ручным и автоматическим методом. Материалы и методы. Исследовались цитологические мазки, приготовленные по методу жидкостной цитологии на приборе Becton Dickinson (технология SurePath), иммуноцитохимическое исследование проводилось на автоматическом иммуностейнере Ventana BenchMark Ultra с использованием коммерческого набора CINtec (определение коэкспрессии р16/Ki-67). Изучено 100 цитологических препаратов (50 пар ПАП-мазок–иммуноцитохимический препарат). Диагностический набор просмотрен пятью цитологами независимо друг от друга, цитологические препараты оценены с помощью 4 категорий по системе Бетесда (2014 г.) и по категориям норма/патология. Оценка коэкспрессии p16/Ki-67 проводилась по рекомендациям производителя (Roch) ручным методом (световой микроскоп) и с помощью автоматической системы Vision Cyto Pap ICC. Статистическая обработка полученных результатов проводилась с помощью пакета программного обеспечения SPSS версии 26.0.0.0 с определением показателей воспроизводимости каппа Коэна и каппа Флейса. Результаты. При оценке воспроизводимости четырех категорий цитологических диагнозов по системе Бетесда (2014 г.) выявлено, что уровень коэффициента каппа Коэна составил 0,048–0,265. Общая каппа Флейса между всеми цитологами составила 0,103. При выделении только двух категорий (норма и патология) уровень воспроизводимости составил от 0,058 до 0,377. При оценке коэкспрессии р16 и Ki-67 выявлена воспроизводимость по каппе Коэна от 0,196 до 0,574, при этом общая каппа Флейса составила 0,407. При сопоставлении результатов оценки каждым из цитологов с нейросетью воспроизводимость по каппе Коэна составила от 0,103 до 0,436. Заключение. В настоящем исследовании воспроизводимость цитологических заключений с применением системы Бетесда 2014 г., а также двух категорий (норма/патология) на основании исследования ПАП-мазков оказалась низкой. С нашей точки зрения, такие результаты прежде всего связаны с большим количеством патологических мазков, включенных в исследование. Применение иммуноцитохимического метода позволило в 3 раза увеличить воспроизводимость диагнозов, что свидетельствует о необходимости использования определения коэкспрессии р16 и Ki-67 для повышения чувствительности и специфичности цитологического метода. Сопоставимые данные воспроизводимости при сравнении ручной и автоматической оценки «двойной метки» свидетельствуют о том, что в настоящее время нейросетевой алгоритм может помочь в области поддержки принятия решения, а не заменить цитолога на диагностическом этапе.
Ключевые слова: ПАП-мазок, коэкспрессия p16/Ki-67, нейросетевой анализ, воспроизводимость, система Бетесда
________________________________________________
Aim. To assess the reproducibility of cytological diagnoses in evaluating liquid cervical smears and immunocytochemical co-expression of p16/Ki-67 using manual and automatic methods. Materials and methods. Cytological smears prepared using the liquid cytology method on the Becton Dickinson device (SurePath technology) were studied. An immunocytochemical study was carried out using a Ventana BenchMark Ultra automatic immunostainer with a commercial CINtec kit (determination of p16/Ki-67 co-expression). In total, 100 cytological slides (50 pairs of Pap-smears and immunocytochemical slides) were studied. The diagnostic kit was reviewed by five cytologists independently, and the cytologic slides were evaluated using four categories according to the Bethesda system (2014) and according to the categories of normal/abnormal. The co-expression of p16/Ki-67 was assessed per the manufacturer's recommendations (Roche) using the manual method (light microscope) and the automatic Vision Cyto Pap ICC system. Statistical processing of the results was performed using the SPSS software package version 26.0.0.0 with the calculation of the reproducibility indices of Cohen's kappa and Fleiss' kappa. Results. When assessing the reproducibility of four categories of cytological diagnoses according to the Bethesda system (2014), Cohen's kappa was 0.048–0.265. The overall Fleiss' kappa between all cytologists was 0.103. When only two categories (normal/abnormal) were used, the reproducibility ranged from 0.058 to 0.377. When assessing the co-expression of p16 and Ki-67, Cohen's kappa reproducibility was from 0.196 to 0.574, while the overall Fleiss' kappa was 0.407. When comparing the evaluation results of each of the cytologists with the neural network, Cohen's kappa reproducibility ranged from 0.103 to 0.436. Conclusion. The reproducibility of cytological diagnoses according to the Bethesda system (2014) and two categories (normal/abnormal) based on the Pap smear study was low. Such results are primarily due to a large number of abnormal smears in the study. The immunocytochemical method has diagnosis reproducibility three times higher, indicating the need to measure the co-expression of p16 and Ki-67 to increase the sensitivity and specificity of the cytological method. Similar reproducibility when comparing the manual and automatic evaluation of the "double label" suggests that the neural network algorithm can currently help in decision support rather than replace the cytologist at the diagnostic stage.
Keywords: Pap smear, p16/Ki-67 co-expression, neural network analysis, reproducibility, Bethesda system
1. Siegel RL, Miller KD, Fuchs HE. Cancer statistics, 2022. CA Cancer J Clin. 2022;72(1):7-33.
2. Boring CC, Squires TS, Tong T. Cancer statistics, 1999. CA Cancer J Clin. 1999;49(1):8-31.
3. Stoler MH, Schiffman M. Interobserver reproducibility of cervical cytologic and histologic interpretations: realistic estimates from the ASCUS-LSIL Triage Study. JAMA. 2001;285(11):1500-5.
4. Selvaggi SM. Implications of low diagnostic reproducibility of cervical cytologic and histologic diagnoses. JAMA. 2001;285(11):1506-8.
5. Hwang H, Follen M, Guillaud M, et al. Cervical cytology reproducibility and associated clinical and demographic factors. Diagn Cytopathol. 2020;48(1):35-42.
6. Kloboves Prevodnik V, Jerman T, Nolde N, et al. Interobserver variability and accuracy of p16/Ki-67 dual immunocytochemical staining on conventional cervical smears. Diagn Cytopathol. 2019;14(1):48.
7. Wentzensen N, Lahrmann B, Clarke MA, et al. Accuracy and Efficiency of Deep-Learning–Based Automation of Dual Stain Cytology in Cervical Cancer Screening. J Natl Cancer Inst. 2021;113(1):72-9.
8. Dey P. Artificial neural network in diagnostic cytology. CytoJournal. 2022;19:146.
9. Sanyal P, Barui S, Deb P, Sharma HC. Performance of A Convolutional Neural Network in Screening Liquid Based Cervical Cytology Smears. J Cytol. 2019;36(3):146.
10. Mohammed MA, Abdurahman F, Ayalew YA. Single-cell conventional pap smear image classification using pre-trained deep neural network architectures. BMC Biomed Eng. 2021;3(1):1-8.
11. Zhang L, Lu L, Member S, et al. DeepPap: Deep Convolutional Networks for Cervical Cell Classification. IEEE J Biomed Health Inform. 2017;21(6):1633-43.
12. Фириченко С.В., Манухин И.Б., Роговская С.И., Манухина Е.И. «Подводные камни» цервикального скрининга. Доктор.Ру. 2018;2(146):26-34 [Firichenko SV, Manukhin IB, Rogovskaya SI, Manukhina ЕI. Pitfalls in Cervical Screening. Doctor.Ru. 2018;2(146):26-34 (in Russian)].
13. Landis JR, Koch GG. The Measurement of Observer Agreement for Categorical Data. Biometrics. 1977;33(1):159.
14. Swailes AL, Hossler CE, Kesterson JP. Pathway to the Papanicolaou smear: The development of cervical cytology in twentieth-century America and implications in the present day. Gynecol Oncol. 2019;154(1):3-7.
15. Salehiniya H, Momenimovahed Z, Allahqoli L, et al. Factors related to cervical cancer screening among Asian women. Riv Eur Sci Med Farmacol. 2021;25(19):6109-22.
16. Cudjoe J, Nkimbeng M, Turkson-Ocran RA, et al. Understanding the Pap Testing Behaviors of African Immigrant Women in Developed Countries: A Systematic Review. J Immigr Minor Health. 2021;23(4):840-56.
17. Chin SS, Jamonek Jamhuri NAB, Hussin N, et al. Factors influencing pap smear screening uptake among women visiting outpatient clinics in Johor. Malays Fam Physician. 2022;17(2):46-55.
18. Settakorn J, Rangdaeng S, Preechapornkul N, et al. Interobserver reproducibility with LiquiPrep liquid-based cervical cytology screening in a developing country. Asian Pac J Cancer Prev. 2008;9(1):92-6.
19. Strander B, Andersson-Ellström A, Milsom I, et al. Liquid-based cytology versus conventional Papanicolaou smear in an organized screening program: a prospective randomized study. Cancer. 2007;111(5):285-91.
20. Nishio H, Iwata T, Nomura H, et al. Liquid-based cytology versus conventional cytology for detection of uterine cervical lesions: a prospective observational study. Jpn J Clin Oncol. 2018;48(6):522-8.
21. Sharma P, Gupta P, Gupta N, et al. Evaluation of the Performance of CinTec® PLUS in SurePathTM Liquid-Based Cervico-Vaginal Samples. Turk Patoloji Dergisi. 2021;37(1):32-8.
22. Сухих Г.Т, Прилепская В.Н., Асатурова А.В., и др. Диагностика, лечение и профилактика цервикальных интраэпительных неоплазий. М., 2020 [Sukhikh GT, Prilepskaia VN, Asaturova AV, et al. Diagnostika, lecheniie i profilaktika tservikal'nykh intraepitel'nykh neoplazii. Moscow, 2020 (in Russian)].
23. Sharma J, Toi P, Siddaraju N, et al. A comparative analysis of conventional and SurePath liquid-based cervicovaginal cytology: A study of 140 cases. J Cytol. 2016;33(2):80-4.
24. Confortini M, Bondi A, Cariaggi MP, et al. Interlaboratory reproducibility of liquid-based equivocal cervical cytology within a randomized controlled trial framework. Diagn Cytopathol. 2007;35(9):541-4.
25. Sriamporn S, Kritpetcharat O, Nieminen P, et al. Kritpetcharat O. Consistency of cytology diagnosis for cervical cancer between two laboratories. Asian Pac J Cancer Prev. 2005;6(2):208-12.
26. Tjalma WAA. Diagnostic performance of dual-staining cytology for cervical cancer screening: A systematic literature review. Eur J Obstet Gynecol Reprod Biol. 2017;210:275-80.
27. Bergeron C, Ikenberg H, Sideri M, et al. Prospective evaluation of p16/Ki-67 dual-stained cytology for managing women with abnormal Papanicolaou cytology: PALMS study results. Cancer Cytopathol. 2015;123(6):373-81.
28. Li Y, Fu Y, Cheng B, et al. A Comparative Study on the Accuracy and Efficacy Between Dalton and CINtec® PLUS p16/Ki-67 Dual Stain in Triaging HPV-Positive Women. Front Oncol. 2022;11:815213.
29. Han Q, Guo H, Geng L, Wang Y. p16/Ki-67 dual-stained cytology used for triage in cervical cancer opportunistic screening. Chin J Cancer. 2020;32(2):208.
30. McMenamin M, McKenna M, McDowell A, et al. Intra- and inter-observer reproducibility of CINtec® PLUS in ThinPrep® cytology preparations. Cytopathology. 2017;28(4):284-90.
31. Wentzensen N, Fetterman B, Tokugawa D, et al. Interobserver reproducibility and accuracy of p16/Ki-67 dual-stain cytology in cervical cancer screening. Cancer Cytopathol. 2014;122(12):914-20.
32. Goh ST, TayKah T, Lim L. Inter-observer Variabilty of CINtec PLUS Dual Staining for p16/ki67. J Am Soc Cytopathol. 2017;6(5):S30.
33. Hammer A, Gustafson LW, Christensen PN, et al. Implementation of p16/Ki67 dual stain cytology in a Danish routine screening laboratory: Importance of adequate training and experience. Cancer Med. 2020;9(21):8235-42.
34. Benevolo M, Allia E, Gustinucci D, Montaguti A. Interobserver reproducibility of cytologic p16INK4a /Ki-67 dual immunostaining in human papillomavirus-positive women. Cancer Cytopathol. 2017;125(3):212-20.
35. Benevolo M, Mancuso P, Allia E, et al. Interlaboratory concordance of p16/Ki-67 dual-staining interpretation in HPV-positive women in a screening population. Cancer Cytopathol. 2020;128(5):323-32.
36. Sornapudi S, Addanki R, Stanley RJ, et al. Automated Cervical Digitized Histology Whole-Slide Image Analysis Toolbox. J Pathol Informatics. 2021;12(1):26.
37. Kanavati F, Hirose N, Ishii T, et al. A Deep Learning Model for Cervical Cancer Screening on Liquid-Based Cytology Specimens in Whole Slide Images. Cancers. 2022;14(5):1159.
38. Özcan Z, Kimiloğlu E, Akyildiz İğdem A, Erdoğan N. Comparison of the Diagnostic Utility of Manual Screening and the ThinPrep Imaging System in Liquid-Based Cervical Cytology. Turk Patoloji Dergisi. 2020;36(2):135-41.
39. Nuttall DS, Hillier S, Clayton HR, et al. A retrospective validation of the FocalPoint GS slide profiler NFR technology by analysis of interval disease outcomes compared with manual cytology. Cancer Cytopathol. 2019;127(4):240-6.
________________________________________________
1. Siegel RL, Miller KD, Fuchs HE. Cancer statistics, 2022. CA Cancer J Clin. 2022;72(1):7-33.
2. Boring CC, Squires TS, Tong T. Cancer statistics, 1999. CA Cancer J Clin. 1999;49(1):8-31.
3. Stoler MH, Schiffman M. Interobserver reproducibility of cervical cytologic and histologic interpretations: realistic estimates from the ASCUS-LSIL Triage Study. JAMA. 2001;285(11):1500-5.
4. Selvaggi SM. Implications of low diagnostic reproducibility of cervical cytologic and histologic diagnoses. JAMA. 2001;285(11):1506-8.
5. Hwang H, Follen M, Guillaud M, et al. Cervical cytology reproducibility and associated clinical and demographic factors. Diagn Cytopathol. 2020;48(1):35-42.
6. Kloboves Prevodnik V, Jerman T, Nolde N, et al. Interobserver variability and accuracy of p16/Ki-67 dual immunocytochemical staining on conventional cervical smears. Diagn Cytopathol. 2019;14(1):48.
7. Wentzensen N, Lahrmann B, Clarke MA, et al. Accuracy and Efficiency of Deep-Learning–Based Automation of Dual Stain Cytology in Cervical Cancer Screening. J Natl Cancer Inst. 2021;113(1):72-9.
8. Dey P. Artificial neural network in diagnostic cytology. CytoJournal. 2022;19:146.
9. Sanyal P, Barui S, Deb P, Sharma HC. Performance of A Convolutional Neural Network in Screening Liquid Based Cervical Cytology Smears. J Cytol. 2019;36(3):146.
10. Mohammed MA, Abdurahman F, Ayalew YA. Single-cell conventional pap smear image classification using pre-trained deep neural network architectures. BMC Biomed Eng. 2021;3(1):1-8.
11. Zhang L, Lu L, Member S, et al. DeepPap: Deep Convolutional Networks for Cervical Cell Classification. IEEE J Biomed Health Inform. 2017;21(6):1633-43.
12. Firichenko SV, Manukhin IB, Rogovskaya SI, Manukhina ЕI. Pitfalls in Cervical Screening. Doctor.Ru. 2018;2(146):26-34 (in Russian).
13. Landis JR, Koch GG. The Measurement of Observer Agreement for Categorical Data. Biometrics. 1977;33(1):159.
14. Swailes AL, Hossler CE, Kesterson JP. Pathway to the Papanicolaou smear: The development of cervical cytology in twentieth-century America and implications in the present day. Gynecol Oncol. 2019;154(1):3-7.
15. Salehiniya H, Momenimovahed Z, Allahqoli L, et al. Factors related to cervical cancer screening among Asian women. Riv Eur Sci Med Farmacol. 2021;25(19):6109-22.
16. Cudjoe J, Nkimbeng M, Turkson-Ocran RA, et al. Understanding the Pap Testing Behaviors of African Immigrant Women in Developed Countries: A Systematic Review. J Immigr Minor Health. 2021;23(4):840-56.
17. Chin SS, Jamonek Jamhuri NAB, Hussin N, et al. Factors influencing pap smear screening uptake among women visiting outpatient clinics in Johor. Malays Fam Physician. 2022;17(2):46-55.
18. Settakorn J, Rangdaeng S, Preechapornkul N, et al. Interobserver reproducibility with LiquiPrep liquid-based cervical cytology screening in a developing country. Asian Pac J Cancer Prev. 2008;9(1):92-6.
19. Strander B, Andersson-Ellström A, Milsom I, et al. Liquid-based cytology versus conventional Papanicolaou smear in an organized screening program: a prospective randomized study. Cancer. 2007;111(5):285-91.
20. Nishio H, Iwata T, Nomura H, et al. Liquid-based cytology versus conventional cytology for detection of uterine cervical lesions: a prospective observational study. Jpn J Clin Oncol. 2018;48(6):522-8.
21. Sharma P, Gupta P, Gupta N, et al. Evaluation of the Performance of CinTec® PLUS in SurePathTM Liquid-Based Cervico-Vaginal Samples. Turk Patoloji Dergisi. 2021;37(1):32-8.
22. Sukhikh GT, Prilepskaia VN, Asaturova AV, et al. Diagnostika, lecheniie i profilaktika tservikal'nykh intraepitel'nykh neoplazii. Moscow, 2020 (in Russian).
23. Sharma J, Toi P, Siddaraju N, et al. A comparative analysis of conventional and SurePath liquid-based cervicovaginal cytology: A study of 140 cases. J Cytol. 2016;33(2):80-4.
24. Confortini M, Bondi A, Cariaggi MP, et al. Interlaboratory reproducibility of liquid-based equivocal cervical cytology within a randomized controlled trial framework. Diagn Cytopathol. 2007;35(9):541-4.
25. Sriamporn S, Kritpetcharat O, Nieminen P, et al. Kritpetcharat O. Consistency of cytology diagnosis for cervical cancer between two laboratories. Asian Pac J Cancer Prev. 2005;6(2):208-12.
26. Tjalma WAA. Diagnostic performance of dual-staining cytology for cervical cancer screening: A systematic literature review. Eur J Obstet Gynecol Reprod Biol. 2017;210:275-80.
27. Bergeron C, Ikenberg H, Sideri M, et al. Prospective evaluation of p16/Ki-67 dual-stained cytology for managing women with abnormal Papanicolaou cytology: PALMS study results. Cancer Cytopathol. 2015;123(6):373-81.
28. Li Y, Fu Y, Cheng B, et al. A Comparative Study on the Accuracy and Efficacy Between Dalton and CINtec® PLUS p16/Ki-67 Dual Stain in Triaging HPV-Positive Women. Front Oncol. 2022;11:815213.
29. Han Q, Guo H, Geng L, Wang Y. p16/Ki-67 dual-stained cytology used for triage in cervical cancer opportunistic screening. Chin J Cancer. 2020;32(2):208.
30. McMenamin M, McKenna M, McDowell A, et al. Intra- and inter-observer reproducibility of CINtec® PLUS in ThinPrep® cytology preparations. Cytopathology. 2017;28(4):284-90.
31. Wentzensen N, Fetterman B, Tokugawa D, et al. Interobserver reproducibility and accuracy of p16/Ki-67 dual-stain cytology in cervical cancer screening. Cancer Cytopathol. 2014;122(12):914-20.
32. Goh ST, TayKah T, Lim L. Inter-observer Variabilty of CINtec PLUS Dual Staining for p16/ki67. J Am Soc Cytopathol. 2017;6(5):S30.
33. Hammer A, Gustafson LW, Christensen PN, et al. Implementation of p16/Ki67 dual stain cytology in a Danish routine screening laboratory: Importance of adequate training and experience. Cancer Med. 2020;9(21):8235-42.
34. Benevolo M, Allia E, Gustinucci D, Montaguti A. Interobserver reproducibility of cytologic p16INK4a /Ki-67 dual immunostaining in human papillomavirus-positive women. Cancer Cytopathol. 2017;125(3):212-20.
35. Benevolo M, Mancuso P, Allia E, et al. Interlaboratory concordance of p16/Ki-67 dual-staining interpretation in HPV-positive women in a screening population. Cancer Cytopathol. 2020;128(5):323-32.
36. Sornapudi S, Addanki R, Stanley RJ, et al. Automated Cervical Digitized Histology Whole-Slide Image Analysis Toolbox. J Pathol Informatics. 2021;12(1):26.
37. Kanavati F, Hirose N, Ishii T, et al. A Deep Learning Model for Cervical Cancer Screening on Liquid-Based Cytology Specimens in Whole Slide Images. Cancers. 2022;14(5):1159.
38. Özcan Z, Kimiloğlu E, Akyildiz İğdem A, Erdoğan N. Comparison of the Diagnostic Utility of Manual Screening and the ThinPrep Imaging System in Liquid-Based Cervical Cytology. Turk Patoloji Dergisi. 2020;36(2):135-41.
39. Nuttall DS, Hillier S, Clayton HR, et al. A retrospective validation of the FocalPoint GS slide profiler NFR technology by analysis of interval disease outcomes compared with manual cytology. Cancer Cytopathol. 2019;127(4):240-6.
ФГБУ «Национальный медицинский исследовательский центр акушерства, гинекологии и перинатологии им. акад. В.И. Кулакова» Минздрава России, Москва, Россия
*a.asaturova@gmail.com
________________________________________________
Anna V. Tregubova, Nadezda S. Tevrukova, Larisa S. Ezhova, Marina V. Shamarakova, Alina S. Badlaeva, Darya A. Dobrovolskaya, Guldana R. Bayramova, Niso M. Nazarova, Alexey Yu. Shilyaev, Aleksandra V. Asaturova*
Kulakov National Medical Research Center for Obstetrics, Gynecology and Perinatology, Moscow, Russia
*a.asaturova@gmail.com