Advertisement

Simultaneous detection and characterization of diminutive polyps with the use of artificial intelligence during colonoscopy

      Abbreviations:

      AI (artificial intelligence), NBI (narrow-band imaging)
      The application of artificial intelligence (AI) in colonoscopy is attracting a growing amount of attention because it has the potential to improve the quality of colonoscopy.
      • Byrne M.F.
      • Shahidi N.
      • Rex D.K.
      Will computer-aided detection and diagnosis revolutionize colonoscopy?.
      • Mori Y.
      • Kudo S.E.
      • Berzin T.M.
      • et al.
      Computer-aided diagnosis for colonoscopy.
      The main focuses of research in this field comprise automated polyp detection
      • Fernandez-Esparrach G.
      • Bernal J.
      • Lopez-Ceron M.
      • et al.
      Exploring the clinical potential of an automatic colonic polyp detection method based on the creation of energy maps.
      • Urban G.
      • Tripathi P.
      • Alkayali T.
      • et al.
      Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy.
      and characterization
      • Byrne M.F.
      • Chapados N.
      • Soudan F.
      • et al.
      Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model.
      • Chen P.J.
      • Lin M.C.
      • Lai M.J.
      • et al.
      Accurate classification of diminutive colorectal polyps using computer-aided analysis.
      (ie, pathologic prediction), which may respectively contribute to a higher rate of adenoma detection and a reduction of the costs related to unnecessary polypectomy. However, there has not yet been any report of technology capable of simultaneous polyp detection and characterization, which is the optimal situation for fully automated colonoscopic observation.

      Description of technology

      We developed a novel AI system that enables automated polyp detection followed by immediate polyp characterization in a real-time fashion (Fig. 1) by use of an endocytoscope (CF-H290ECI; Olympus Corp, Tokyo, Japan). The endocytoscope has the special capability of 520-fold magnification power in addition to normal endoscope functions (Figs. 2 and 3).
      Figure thumbnail gr1
      Figure 1Real-time computer-aided diagnostic system designed for use with endocytoscopes (EB-00 and EB-01 prototypes from Cybernet System Corp, Tokyo, Japan).
      Figure thumbnail gr2
      Figure 2The endocytoscope (CF-H290ECI; Olympus Corp, Tokyo, Japan) used with the novel artificial intelligence system.
      Figure thumbnail gr3
      Figure 3The tip of the endocytoscope. It has the function of a contact microscopy system with 520-fold magnification power, and it also works as an ordinary high-definition colonoscope.
      The software used for automated polyp detection (EB-00 prototype; Cybernet System Corp, Tokyo, Japan) was designed to analyze the nonmagnified white-light, high-definition, endoscopic video in real time based on a convolutional 3-dimensional network, which is a type of deep learning methodology.
      • Misawa M.
      • Kudo S.E.
      • Mori Y.
      • et al.
      Artificial intelligence-assisted polyp detection for colonoscopy: initial experience.
      The AI system uses this algorithm to automatically detect polyps, and it indicates their presence by outputting a yellow color in the 4 corners of the displayed image and emitting a warning sound (Fig. 4). The automated detection system is able to recognize the presence of the polyps even if there are multiple lesions in the monitor. A pilot study showed that the AI system detected 94% of the test polyps (47 of 50), with a false-positive detection rate of 60% (51 of 85).
      • Misawa M.
      • Kudo S.E.
      • Mori Y.
      • et al.
      Artificial intelligence-assisted polyp detection for colonoscopy: initial experience.
      Figure thumbnail gr4
      Figure 4The artificial intelligence system indicates the automatic detection of a flat adenoma (3 mm, 0-IIa) in the ascending colon by outputting a yellow color in the 4 corners of the display screen and emitting a warning sound. The probability of the presence of the polyp is displayed in the upper-left corner of the endoscopic image.
      The software used for automated polyp characterization (EB-01 prototype; Cybernet System Corp) is designed to predict the pathologic features of the detected polyp as either neoplastic or non-neoplastic after acquisition of the in vivo macroscopic image obtained by the endocytoscope. A probability of the predicted pathologic features is also outputted. The algorithm is based on texture analysis powered by a support vector machine, which is a type of hand-crafted analytic method.
      • Mori Y.
      • Kudo S.E.
      • Misawa M.
      • et al.
      Real-time use of artificial intelligence in identification of diminutive polyps during colonoscopy: a prospective study.
      The AI system is able to analyze microvascular images of the polyp obtained by endocytoscopic observation (at 520-fold magnification) with the narrow-band imaging (NBI) mode (Olympus Corp). A large-scale prospective study
      • Mori Y.
      • Kudo S.E.
      • Misawa M.
      • et al.
      Real-time use of artificial intelligence in identification of diminutive polyps during colonoscopy: a prospective study.
      showed that this software has negative predictive values for identifying diminutive rectosigmoid adenomas of 95.2% with the NBI mode, which exceeds the threshold required for the “diagnose-and-leave hyperplastic polyps” strategy.
      • Rex D.K.
      • Kahi C.
      • O'Brien M.
      • et al.
      The American Society for Gastrointestinal Endoscopy PIVI (Preservation and Incorporation of Valuable Endoscopic Innovations) on real-time endoscopic assessment of the histology of diminutive colorectal polyps.
      The newly developed, comprehensive AI system combines the abovementioned 2 algorithms and is directly connected to a standard endoscopy unit (Evis Lucera Elite; CV-290, Olympus Corp) that allows fully automated interpretation of the polyps encountered during ongoing colonoscopy. The latencies of the detection and characterization system are just 0.1 to 0.5 seconds after the polyp is detected, and 0.4 seconds after the capture button of the endocytoscope is pushed, which enables endoscopists to make real-time decisions regarding the necessity of polypectomy. However, the AI system has several limitations: it is not designed to detect dysplasia in patients with inflammatory bowel disease, and it is unable to diagnose sessile serrated adenomas/polyps, which might be misdiagnosed as non-neoplastic.
      Regarding the clinical implementation of this technology, we are still evaluating the automated polyp detection system in an experimental setting, but we have submitted the regulatory approval application for the automated polyp characterization system in Japan. The endocytoscope is already commercially available.

      Case presentations

      These procedures were prospectively done after approval was obtained from the local ethical committee and informed consent was secured from the patients.
      In patient 1, a diminutive adenoma (3 mm, 0-IIa) in the ascending colon was automatically detected by the AI system (Fig. 4). After detection of the polyp, the endoscopist obtained a microvascular image by contacting the polyp with the tip of the endocytoscope using the NBI mode. The endoscopist subsequently pushed the capture button of the endocytoscope, and the second algorithm was immediately activated and predicted the polyp pathologic features as neoplastic (Fig. 5) (Video 1, available online at www.VideoGIE.org).
      Figure thumbnail gr5
      Figure 5The artificial intelligence system automatically outputs the predicted pathologic features of the detected polyp as neoplastic with a probability of 90% immediately after the endoscopist pushes the capture button of the endocytoscope. The polyp shown in this figure is the same one described in .
      In patient 2, a diminutive hyperplastic polyp (3 mm, 0-IIa) in the rectum was automatically detected and characterized as non-neoplastic by the AI system (Figs. 6 and 7), by use of a technique similar to that described for patient 1.
      Figure thumbnail gr6
      Figure 6The artificial intelligence system indicates the automatic detection of a flat hyperplastic polyp (3 mm, 0-IIa) in the rectum by outputting a yellow color in the 4 corners of the display screen and emitting a warning sound.
      Figure thumbnail gr7
      Figure 7The artificial intelligence system automatically outputs the predicted pathologic features of the detected polyp as non-neoplastic with a probability of 89% immediately after the endoscopist pushes the capture button of the endocytoscope. The polyp shown in this figure is the same one described in .
      An additional 4 patients are presented in the video, as detailed in the legend for Video 1.

      Disclosure

      Drs Y. Mori, Kudo, and Misawa are recipients of speaking honoraria from Olympus and inventors’ premiums for the patent from Showa Univ. Dr K. Mori is the recipient of research funding from Cybernet.

      Acknowledgments

      We thank Dr Hayato Itoh, PhD, and Dr Masahiro Oda, PhD, of the Graduate School of Informatics, Nagoya University, for their invaluable contributions to the present study. We thank Dr Kelly Zammit, BVSc, of the Edanz Group (www.edanzediting.com/ac) for editing a draft of this manuscript. The present study was supported by the Project on Utilizing High-Definition Medical Imaging Data up to 8K Quality from the Japan Agency for Medical Research and Development (AMED) and Japan Society for the Promotion of Science (JSPS) Grant-in-Aid for Scientific for Research (KAKENHI) grant number 17H05305.

      Supplementary data

      • Loading ...

      References

        • Byrne M.F.
        • Shahidi N.
        • Rex D.K.
        Will computer-aided detection and diagnosis revolutionize colonoscopy?.
        Gastroenterology. 2017; 153: 1460-1464e1
        • Mori Y.
        • Kudo S.E.
        • Berzin T.M.
        • et al.
        Computer-aided diagnosis for colonoscopy.
        Endoscopy. 2017; 49: 813-819
        • Fernandez-Esparrach G.
        • Bernal J.
        • Lopez-Ceron M.
        • et al.
        Exploring the clinical potential of an automatic colonic polyp detection method based on the creation of energy maps.
        Endoscopy. 2016; 48: 837-842
        • Urban G.
        • Tripathi P.
        • Alkayali T.
        • et al.
        Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy.
        Gastroenterology. 2018; 155: 1069-1078
        • Byrne M.F.
        • Chapados N.
        • Soudan F.
        • et al.
        Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model.
        Gut. Epub 2017 Oct 24;
        • Chen P.J.
        • Lin M.C.
        • Lai M.J.
        • et al.
        Accurate classification of diminutive colorectal polyps using computer-aided analysis.
        Gastroenterology. 2018; 154: 568-575
        • Misawa M.
        • Kudo S.E.
        • Mori Y.
        • et al.
        Artificial intelligence-assisted polyp detection for colonoscopy: initial experience.
        Gastroenterology. 2018; 154: 2027-2029 e3
        • Mori Y.
        • Kudo S.E.
        • Misawa M.
        • et al.
        Real-time use of artificial intelligence in identification of diminutive polyps during colonoscopy: a prospective study.
        Ann Intern Med. 2018; 169: 357-366
        • Rex D.K.
        • Kahi C.
        • O'Brien M.
        • et al.
        The American Society for Gastrointestinal Endoscopy PIVI (Preservation and Incorporation of Valuable Endoscopic Innovations) on real-time endoscopic assessment of the histology of diminutive colorectal polyps.
        Gastrointest Endosc. 2011; 73: 419-422