Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

In vivo detection of plaque erosion by intravascular optical coherence tomography using artificial intelligence

Open Access Open Access

Abstract

Plaque erosion is one of the most common underlying mechanisms for acute coronary syndrome (ACS). Optical coherence tomography (OCT) allows in vivo diagnosis of plaque erosion. However, challenge remains due to high inter- and intra-observer variability. We developed an artificial intelligence method based on deep learning for fully automated detection of plaque erosion in vivo, which achieved a recall of 0.800 ± 0.175, a precision of 0.734 ± 0.254, and an area under the precision-recall curve (AUC) of 0.707. Our proposed method is in good agreement with physicians, and can help improve the clinical diagnosis of plaque erosion and develop individualized treatment strategies for optimal management of ACS patients.

© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

Acute coronary syndrome (ACS) remains a major cause of morbidity and mortality worldwide [1]. Intracoronary thrombus caused by plaque rupture and erosion are recognized as the main pathological mechanisms of ACS [2]. Previous studies reported that about 35% of thrombotic sudden coronary deaths and 25% of acute myocardial infarction were caused by plaque erosion [3,4]. Different from plaque rupture, plaque erosion usually occurs over pathological intimal thickening, fibrous plaque, or lipid plaque with intact fibrous caps which are rich in smooth muscle cells and proteoglycans, and lack for inflammatory cells such as macrophages and T cells [5]. Pathologically, plaque erosion is defined as the loss of endothelial cells in the absence of fibrous cap ruptures [6], and is typically characterized by insignificant coronary narrowing and residual thrombus due to apoptosis or denudation of superficial endothelial cells and is rich in platelets but less in fibrin [7,8]. Recent studies have shown that patients with plaque erosion might be treated conservatively with anti-thrombotic therapy without coronary stenting [9,10]. Therefore, the identification and analysis of plaque erosion are of great significance for developing the optimal treatment strategy.

Optical coherence tomography (OCT) is a light-based interferometric imaging modality that can provide three-dimensional depth-resolved microstructure of coronary arteries with micron-level resolution [11]. OCT is currently the only clinically available imaging modality that can diagnose plaque erosion in vivo [12]. In 2013, Jia et al. for the first time characterized and categorized the OCT-defined plaque erosion in vivo as definite OCT-erosion and probable OCT-erosion [13]. Definite OCT-erosion is defined as an intact fibrous cap with thrombus and clearly visualized underlying plaque structure. Probable OCT-erosion is either an intact fibrous cap with no thrombus in and around the culprit lesion and with irregular luminal surface, or there is thrombus at the culprit lesion site resulting in obscuring of the underlying plaque structure, without superficial lipid pool or calcification found proximal or distal to the thrombus [13]. Figure 1 shows plaque erosion (Fig. 1(a)) with a comparison with typical fibrous (Fig. 1(b)), lipid (Fig. 1(c)), and calcified plaques (Fig. 1(d)). More than half of plaque erosion occur on fibrous plaques, and about 40% occur on stable lipid plaques with small lipid cores and intact thick fibrous caps, while calcified plaques are less common [14]. Additional examples of plaque erosion can be found in [15].

 figure: Fig. 1.

Fig. 1. Plaque erosion and typical atherosclerotic plaques on intravascular OCT images. (a) Plaque erosion. (b) Fibrous plaque from the same vessel 9.8 mm proximal from the erosion site in Fig. 1(a). (c) Lipid plaque. (d) Calcified plaque.

Download Full Size | PDF

Currently, the challenges of real-time in-procedure diagnosis and decision making, the difficulty of image interpretation along with significant inter-observer variability, and massive image overload (hundreds of images in a pullback) have suggested a high demand for automated image analysis. Artificial intelligence (AI) can be useful to standardize and automate intravascular OCT image analysis. Deep learning is a data-driven AI algorithm with the advantage of automatically and hierarchically extracting features directly from images, which has already made a profound impact on computer vision and image analysis applications, such as image classification [16], image segmentation [17], image completion [18] and so on. Deep learning methods have also shown significant improvement over classic methods [19] for medical image processing tasks such as disease classification [20] and tumor segmentation [21].

There are multiple reports of automated intravascular OCT image analysis using deep learning techniques, mainly including lumen segmentation, plaque characterization, stent analysis, and image based fractional flow reserve (FFR) calculation [22,23]. Miyagawa et al. [24] evaluated three CNN architectures for lumen segmentation. Huang et al. [25] proposed a deep residual lumen segmentation network with multi-scale feature fusion based on attention mechanism and achieved good robustness and accuracy. Li et al. [26] proposed a spatial-temporal encoder-decoder network for segmenting calcified plaques from intravascular OCT pullbacks, which achieved state-of-the-art performance and allowed for comprehensive assessment of coronary calcification for facilitating PCI planning. There are also many studies on automating the assessment of other major types of atherosclerotic plaques, such as fibrous plaques, lipid plaques, etc [2729]. Chu et al. [30] developed and validated an automatic framework for plaque characterization using AI, which is the largest study so far to validate OCT-derived plaque composition by AI. For stent analysis, Guo et al. [31] proposed a local-global refinement network to integrate local-patch information with global content for strut detection, and achieved promising performance on a clinical dataset of 7000 intravascular OCT images. Yang et al. [32] proposed an U-shaped neural network for automated detection of stent struts with both thin and thick tissue coverage, and enabled stent classification and stent area measurement for single and multiple implanted stents. For computational FFR, Ding et al. [33] proposed a novel method for fast computation of FFR from OCT images, termed Optical Flow Ratio (OFR), which employed deep learning as pre-processing steps for image segmentation, and demonstrated that OFR had good diagnostic concordance with FFR wire measurements. Although all of the aforementioned studies are promising, few studies have focused on plaque erosion, where currently the in vivo diagnosis mainly relies on physicians’ visual assessment of intravascular OCT images, which could lead to significant inter-observer variability. Our group previously reported computer-aided detection of plaque erosion [34], where the algorithm was semi-automated requiring the user to draw a region of interest (ROI). Fully automated detection of plaque erosion can significantly reduce inter-observer variability and improve clinical workflow, and is an unmet clinical need.

In this study, we propose a fully automated AI method using convolutional neural networks (CNNs) for the detection of definite OCT-erosion in intravascular OCT images. To enhance plaque erosion detection, we extended the Mask RCNN architecture [35] which achieved leading performance in many computer vision and medical image analysis tasks [36], into a shape-encoded CNN architecture. We further proposed a post-processing algorithm based on domain knowledge to optimize the detection results and improve the algorithm precision. We conducted extensive ablation experiments comparing different CNN architectures, as well as validation studies against human analysts to evaluate the performance of the proposed method.

2. Methods

2.1 Dataset

We collected 83 OCT pullbacks from 83 ACS patients with culprit plaque erosion who had undergone pre-intervention OCT imaging of culprit lesions. All data were acquired at The 2nd Affiliated Hospital of Harbin Medical University (Harbin, China), using a commercially available OCT system (Abbott C7-XR). The dataset of one pullback consisted of hundreds of cross-sectional images, and the distance between adjacent frames is 200 µm. The initial images were acquired and stored in polar representation with a resolution of 976×504 pixels in grayscale, then were transformed into Cartesian coordinates with a size of 1024×1024 pixels before image analysis.

If a vessel had received previous stent implantation, the stented vessel segments were excluded and the remaining images were included for analysis. In total, 29914 cross-sectional images were included for analysis, of which 3275 images were with definite OCT-erosion. Probable OCT-erosion was not assessed in this study, and definite OCT-erosion was simply referred as plaque erosion in subsequent descriptions. One experienced physician (Physician 1) labeled all the plaque erosion using Amira software (Thermo Fisher Scientific). The entire datasets were divided at a ratio of 3:1:1 into a training set containing 51 pullbacks and 18519 images, where 2015 images were with plaque erosion, to train CNN models, a validation set containing 16 pullbacks, 5842 images, where 655 images were with plaque erosion, to optimize the hyperparameters of the model, and a test set containing 16 pullbacks, 5553 images, where 605 images were with plaque erosion, to evaluate the performance of the final model. For training and validation, only the 2670 images with plaque erosion in the training set and validation set were utilized, but all the 5553 images in the test set were used for evaluation. Two other physicians (Physician 2 and 3) only labeled the test set blinded to physician 1, which was used for evaluating inter-observer agreement. In addition, 1600 randomly selected images from the training set were divided into a training and a validation set at a ratio of 3:1 for training a vessel lumen segmentation algorithm for facilitating shape feature extraction employed in the AI algorithm, and 400 randomly selected images from the test set were used for validating the lumen segmentation algorithm.

2.2 Plaque erosion detection algorithm overview

To identify plaque erosion on intravascular OCT images is a challenging task because currently available OCT imaging system does not have sufficient resolution to directly visualize the loss of endothelial cells. Physicians usually utilize two domain knowledge for making a diagnosis: the presence of OCT image features such as superficial mural thrombus and the lack of fibrous cap rupture, and clinical interpretation of plaque erosion with exclusion of other types of plaques. The detection system should preferably be able to provide interpretable information to clinicians for decision making. As shown in Fig. 2, our proposed detection system consists of two components. First, predicting the region (mask) of plaque erosion based on a shape-encoded CNN, which we termed Mask RCNN-CK (Mask RCNN with convexity and curvature). The convexity and curvature features are specifically designed to capture the mural thrombus present in plaque erosion. However, the presence of such feature is not specific to erosion, as plaque rupture, residual blood and thrombus may also have similar features. Second, refine the initial prediction by implementing a classification algorithm to exploit clinically interpretable features of the underlying plaque using a support vector machine (SVM) [37] classifier and a three-dimensional processing module. Next, we introduce the key elements of the plaque erosion detection algorithm.

 figure: Fig. 2.

Fig. 2. Artificial intelligence (AI) based plaque erosion detection algorithm. The original image and two key shape features convexity and curvature are input into a shape-encoded convolutional neural network. The preliminary detected mask is fed to the second module, where post-processing based on domain knowledge is performed to generate the final output.

Download Full Size | PDF

2.3 Shape-encoded convolutional neural network

The shape-encoded CNN model was formulated as a multi-input CNN by combining two shape features convexity and curvature as additional input branches into the Mask RCNN architecture [35]. The details of the framework are presented in the following sections.

Shape features. CNNs are more concerned with textural features of images [38], but shape feature is the more obvious signature of plaque erosion in intravascular OCT. We chose convexity and curvature to express luminal surface irregularity (Fig. 3). In fact, adding the shape features is essential as we found most existing state-of-the-art CNN models struggled for plaque erosion detection (see Table 3). We first segmented the lumen boundary using another state-of-the-art CNN model U-Net [39]. Based on the segmented vessel lumen boundaries, curvature was computed as the rate of change of slope of the lumen boundary curve [40,41], and convexity was computed as the difference between the convex hull of the lumen and the lumen itself [42](Fig. 3).

 figure: Fig. 3.

Fig. 3. Characterization of convexity and curvature of plaque erosion. From left to right are the original image, enlarged view of the lumen, convexity and curvature. Convexity: computed as the difference between the convex hull of the lumen and the lumen itself, labeled in cyan. Curvature: defined as the rate of change of slope of the lumen boundary curve, labeled in a continuous colormap.

Download Full Size | PDF

Tables Icon

Table 1. Comparison of different shape encoder modules

Tables Icon

Table 2. List of the clinically interpretable features

Tables Icon

Table 3. Quantitative comparison with state-of-the-art CNN methods

Network model. The entire network contains three input branches, the original image, the convexity and curvature input (Fig. 4). Convexity and curvature undergo two independent feature extraction modules (see Table 1) to obtain shape feature maps. The main branch features are exacted with a Feature Pyramid Networks (FPN) [43], and are then served as the input of the second stage of the network to generate potential ROIs of erosion via Region Proposal Network (RPN). The rest of the network simply follows the design in Mask RCNN [35].

 figure: Fig. 4.

Fig. 4. Shape-encoded CNN model for plaque erosion detection in intravascular OCT. The original image, convexity and curvature are fed into feature extraction modules to generate ROIs via Region Proposal Network (RPN). The RCNN Head and Mask Head output the detection results.

Download Full Size | PDF

Encoder modules of shape features. The feature extraction modules of convexity and curvature play an essential role to extract the key features of plaque erosion. Representations of shape features are simple binary images with obvious features, which do not require very deep convolutional layers for feature extraction. We designed and compared the VGG module [44], Residual module [35] and Inception module [35]. Table 1 shows the details of the three shape encoder modules with an output size of 256×256 as an example.

2.4 Post-processing based on domain knowledge

Classification using clinically interpretable features. CNNs are the intuitive reflection based on input/output, and there still exists a gap in terms of professional medical domain knowledge compared with human experts. Therefore, clinically interpretable features were further extracted from the output of the shape-encoded CNN, and input into a SVM [37] model for plaque erosion classification. As shown in Fig. 5, the area corresponding to the predicted mask of plaque erosion was extracted as ROIs from the original image based on the output of the shaped-encoded CNN. The clinically interpretable features extracted from each ROI were designed to capture the characteristics of the underlying plaque and help differentiate erosion from other tissues or artifacts.

 figure: Fig. 5.

Fig. 5. Schematic diagram of the classification model using clinically interpretable features. The area corresponding to the mask output from the shape-encoded convolutional neural network in the original image is used to extract clinically interpretable features, which are input into a SVM model to obtain the classification results of plaque erosion.

Download Full Size | PDF

As shown in Table 2, the clinically interpretable features mainly include optical features, morphological features, geometric features and three-dimensional continuity [45]. The optical features include the intensity, gradient, contrast, histogram and optical attenuation of tissue [46,47]. The morphological features include the tissue protrusion area and roughness of the tissue surface. The geometric features were used to prevent the guide wire artifacts from obscuring plaque erosion detection.

Three-dimensional processing module. Based on the three-dimensional continuity of plaque erosion, we further refined the plaque erosion detection results from the CNN. 26-connectivity neighborhood system [48] was used to label 3D connected regions. Any lesion spanning less than 3 frames was removed. Further, if there was any overlap between the connected regions in the consecutive 3 frames, the middle image was labeled with the interpolated intersection between the detected erosion of the previous and the next frames.

2.5 Implementation details

The proposed CNN method (Mask RCNN-CK) was implemented using the Keras 2.2.4 framework based on Tensorflow 1.9.0. We used 4 Nvidia Titan XP GPUs with 12*4 = 48GB memory for training the CNNs. Due to memory constraint, we made the batch size to 4. We set the optimizer to Stochastic Gradient Descent (SGD), the momentum to 0.9, and the loss function to cross entropy. We trained the model with pre-trained weights with MS COCO datasets [49] and an initial learning rate of 0.001. The model was trained in 260 epochs which took around 23h. For lumen segmentation, the implementation details were the same as that in the original publication [39].

The algorithm performance for plaque erosion detection was evaluated on a plaque-level instead of pixel-level, where the detected plaque erosion was classified as true positive (TP), false positive (FP) and false negative (FN) with the physician annotated ground truth as the gold standard. True negative (TN) was not informative for this study since any regions other than plaque erosion fit in this category, and was therefore not collected. Unlike typical image segmentation tasks, plaque erosion usually does not have clear borders except for the part coincident with the lumen, and it is not realistic to precisely delineate the plaque erosion boundary even for physicians. We therefore required the algorithm-detected plaque erosion to have at least 50% overlap with the human annotation to be counted as TP. Specifically, if the Dice Coefficient (DIC) [50] between the detected mask and any of ground truth was higher than 0.5, the predicted plaque erosion was classified as TP. The detection accuracy was evaluated using the two widely accepted metrics precision and recall [51]:

$$Precision = \frac{{TP}}{{TP + FP}}\,\,\,\,\,\, Recall = \frac{{TP}}{{TP + FN}}.$$

Precision-recall (PR) curve was further extracted. Lumen segmentation accuracy was evaluated by DIC on a pixel-level. Results were presented as mean ± standard deviation.

3. Results

3.1 Main results

The proposed AI algorithm was run on the entire intravascular OCT pullbacks with no need to exclude any frames. The running time for fully automated detection of plaque erosion was 0.447s per image for the entire method, where the shape-encoded network Mask RCNN-CK (with Inception module) took 0.202s. Figure 6 shows four examples of automated detection of plaque erosion compared with ground truth. Overall, the algorithm achieved close agreement with ground truth. Due to the fact that intravascular OCT has limited depth penetration and plaque erosion usually does not exhibit lateral boundaries, it is only possible to accurately detect the boundary on the luminal side. There is a close agreement between the results of the proposed network and ground truth for images with plaque erosion (Fig. 6(b)-(d)). For areas without plaque erosion, the proposed network performed well on the reduction of false positives (Fig. 6(a)).

 figure: Fig. 6.

Fig. 6. Plaque erosion detection by the proposed AI algorithm in four cases, with each column showing one case. From top to bottom are the original images, ground truth, and predicted plaque erosion by our proposed method respectively.

Download Full Size | PDF

With the test set consisting of 5553 cross-sectional intravascular OCT images from 16 pullbacks, the proposed AI algorithm achieved 0.600 ± 0.279 precision and 0.949 ± 0.129 recall for plaque erosion (Table 3). The performance of lumen segmentation was 0.980 ± 0.010 in DIC. Comparing with other widely used state-of-the-art CNN methods including Faster RCNN [52], U-Net [39], DeepLabv3+ [35], TransUNet [53], nnUNet [54], V-Net [55] and SOLOv2 [56], Mask RCNN [35] performed relatively better to be selected as the basis network to detect plaque erosion automatically, and the shape-encoded module brought significant enhancement of algorithm precision (Table 3). The post-processing algorithm based on domain knowledge can effectively reduce possible learning bias and improve detection accuracy. It generated a significant visual improvement on the final results, which is reflected numerically by the significant improvement in precision. We also tested the algorithm on additional ten pullbacks consisting of 3224 images without erosion and achieved an accuracy of 0.932 ± 0.075.

The proposed AI algorithm reached an area under the precision-recall curve (AUC) of 0.707 on the test set (Fig. 7). In clinic, different operating points are often required depending on the specific patient management strategies. Based on the algorithm output, we propose operating point A and B. At operating point B, a recall of 95% is achieved at 60% precision, which is the direct result of the algorithm; at operating point A, an 80% recall is achieved at 73% precision (Fig. 7). In this study, we chose the operating point A as the final result to better balance the recall (0.800 ± 0.175) and precision (0.734 ± 0.254).

 figure: Fig. 7.

Fig. 7. Precision-recall curve (orange) shows the performance of the automated prediction of plaque erosion against ground truth. Operating point A (73% precision) and B (95% recall) are indicated by the blue and green triangles respectively. Blue-shaded areas indicate 95% confidence intervals.

Download Full Size | PDF

There are still some failure cases in the proposed AI algorithm. Figure 8 shows four cases in which plaque erosion was not detected, which were agreed by at least 2 out of the 3 physicians. Figure 8(a) and Fig. 8(b) are two challenging examples of plaque erosion with unclear lumen boundaries (Fig. 8(a)), or with relatively low luminal surface irregularity (Fig. 8(b)). For the start or end frame of the eroded segment, plaque erosion is relatively small and less obvious, and the corresponding features are relatively difficult to be detected (Fig. 8(c) and (d)).

 figure: Fig. 8.

Fig. 8. Undetected plaque erosion, with each column showing one case. From top to bottom are the original images and ground truth respectively. The blue circle, green triangle, and pink rhombus on the top right respectively indicates physician 1, physician 2 and physician 3 agreed on the case.

Download Full Size | PDF

We quantitatively compared the differences between the proposed AI algorithm and three physicians on plaque erosion detection using the test set (Table 4). Three physicians independently labeled the same 16 pullbacks in the test set blinded to each other. Physician 1 annotated all pullbacks as plaque erosion. Physician 2 annotated one pullback as plaque rupture rather than erosion, and physician 3 annotated two pullbacks (different with physician 2) as plaque rupture. The recall of the proposed AI algorithm against physician 1 and physician 3 was higher than the agreement between the physicians, but was slightly lower than physician 2. The error in precision is possibly due to the presence of probable OCT-erosion (see Fig. 9). We additionally evaluated the agreement between the proposed method and the consensus of the three physicians which was generated by only keeping the plaque erosion segmented by all three physicians with at least 50% overlap. The algorithm achieved a precision of 0.422 ± 0.250 and a recall of 0.928 ± 0.124 compared with the consensus containing 11 pullbacks and 3612 images.

 figure: Fig. 9.

Fig. 9. Likely probable OCT-plaque erosion detected by the AI algorithm but missed by physicians, with each column showing one case. From top to bottom are the original images and the output from the algorithm respectively.

Download Full Size | PDF

Tables Icon

Table 4. Quantitative comparison between the proposed AI algorithm and three physicians

Figure 9 illustrates some typical examples of probable OCT-erosion identified by the AI algorithm but were not labeled by physicians. In general, only using the features presented by intravascular OCT is difficult to eliminate the false positives shown in Fig. 9(a)-(g), where a fibrous or lipid plaque with an irregular lumen surface is usually present. Figure 9(h) shows local luminal surface irregularity in the absence of thrombus, which was also detected by the algorithm as false positives.

3.2 Ablation study

We run a number of ablation experiments to analyze the proposed CNN model Mask RCNN-CK. Results are shown and discussed in detail below.

The design choice of encoder modules. One concern about the shape-encoded CNN model is the feature extraction modules of shape features. We compared the three different encoder modules (VGG Module, Residual Module, Inception Module) from simple to complex, applying to Mask RCNN with convexity shape encoding (Mask RCNN-C). The results are shown in Table 5, among which Inception module performed better. Also, we evaluated the effect of different feature map sizes (512×512 pixels, 256×256 pixels and 128×128 pixels) with Inception Module on network performance as shown in Table 6. We repeated the above experiments on Mask RCNN-CK, and the best performing configuration was Inception Module with the feature map size of 128×128 pixels on both convexity and curvature, which was used as our final choice.

Tables Icon

Table 5. Quantitative comparison on Mask RCNN with convexity with different encoder modules

Tables Icon

Table 6. Quantitative comparison on Mask RCNN with convexity with different sizes of feature map

The choice of shape features. Another concern about the shape-encoded CNN model is the choice of shape features, and its choice is crucial to the algorithm performance. We proposed two different shape features: convexity characterizes the protrusion of the plaque erosion; curvature represents luminal surface irregularity. We additional evaluated Mask RCNN model with a lumen mask input (Mask RCNN-L) in binary format to implicitly encode the two shape features, and quantitative comparison is shown in Table 7. It can be seen that Mask RCNN-CK performed significantly better than Mask RCNN-L, Mask RCNN-C and Mask RCNN with curvature (Mask RCNN-K) on both precision and recall. This demonstrated that the two shape features are complementary and results are better if they are combined together and used explicitly. We further evaluated simply calculating the shape feature values after the Mask RCNN output for post-CNN classification. We computed the convexity area and the curvature value of the plaque erosion regions of the training set, and a convexity area of 0.0007 mm2 and a curvature value of 0.335 were used as thresholds, with which 97% of the training set were correctly classified as plaque erosion. This model generated a much lower precision of 0.195 ± 0.123 and a recall of 0.959 ± 0.084. This indicates that it is desirable to use the shape features at the first place as direct input branches of the CNN models.

Tables Icon

Table 7. Quantitative comparison on algorithms with different combinations of shape features

4. Discussion

We demonstrated an AI algorithm for automated detection of plaque erosion in vivo by intravascular OCT, which included a plaque erosion detector using a shape-encoded CNN, and a domain knowledge based classifier. While evaluating the performance of the proposed algorithm, we have found seemingly significant inter-observer variability especially in precision (Table 4). This is likely due to the fact that physicians may annotate different cross-sections of the same underlying plaque erosion in the OCT pullback and generated exaggerated numerical differences using our overlapping criteria for evaluating inter-observer agreement. However, such stringent segmentation of plaque erosion is unnecessary in clinic as identifying the presence of plaque erosion as the cause of ACS is more relevant for optimizing individualized precision treatment. Another possible reason may be that there are certain limitations in the detection of plaque erosion by OCT. Despite the superior resolution of OCT, current clinically available OCT systems cannot visualize the absence of endothelial cells on the plaque surface, and the presence of luminal thrombus hampers the penetration of light into the underlying plaque, making reliable measurements and diagnosis of the various causes of coronary thrombus difficult [5759]. In many of these cases, it is likely that thrombus still remained, and this may have led to a misdiagnosis of erosion [60]. The definitions of plaque erosion in OCT were not validated by pathology especially for probable OCT-erosion. Plaque erosion pathologically is not defined as the absence of lipid or calcification at the culprit site. The underlying plaque is fibroatheroma, which may include lipid and is not necessarily lacking calcification. Only the absence of endothelial cells is a key pathological criterion for erosion which is difficult to visualize with OCT. Thus, the accuracy of OCT to identify probable OCT-erosion remains to be validated. Our dataset contained a variety of lesion types commonly encountered in clinic, including plaque erosion, plaque rupture, calcified nodules, and others, as well as abundant images without plaque erosion for testing the false positive rate of the proposed algorithm. Validation studies demonstrated that our proposed method is robust and can achieve compelling performance in the real world clinical dataset.

The proposed AI model is desirable for further clinical translation and applications. The proposed algorithm is fully automated which can provide fast feedback only from the original OCT images without any additional input. Although there are false positives, they have characteristics similar to plaque erosion (Fig. 9) and deserve further attention from clinical input. For these cases, it is obvious that the trained deep learning model identified the key features of plaque erosion, but gave different detection results from physicians because probable OCT-plaque erosion was not assessed in this study. Our results suggest that although the algorithm cannot completely replace physicians for diagnosis, it can automatically provide reliable objective recommendations while effectively reducing inter-observer variability. Human-Machine interaction may be a better way for accurate detection of plaque erosion. In addition, our proposed algorithm generates configurable operating points, which can be changed according to specific diagnostic and treatment needs.

The proposed AI algorithm has important potential values for the diagnosis and treatment of ACS patients. Plaque rupture and plaque erosion are the most common causes of ACS. However, previous in vitro, autopsy and in vivo studies demonstrated that eroded and ruptured plaques are distinct entities with unique pathobiology, morphologies, incidence, and clinical behaviors, suggesting that their optimal treatment may differ [61,62]. Advances in the medical management of atherosclerosis (lipid lowing therapy) leads to an increasing proportion of plaque erosion relative to rupture. If targeted and individualized treatment can be carried out for erosion patients, it may help make a significant impact on the management of ACS patients. The current “2018 European Society of Cardiology/European Society of Cardiothoracic Surgery Guidelines for Myocardial Revascularization” classifies patients with acute myocardial infarction based on the performance of the electrocardiogram, and early percutaneous coronary intervention (PCI) is recommended to obtain early myocardium reperfusion and revascularization [63]. However, stenting in the acute phase in the presence of a large thrombus burden may lead to acute complications (distal embolization, no-reflow phenomenon, and acute stent thrombus) and major adverse cardiac events [64]. Considering the potential differences in the response of ruptured and eroded vessels to PCI and a better long-term prognosis of plaque erosion compared with plaque rupture, the treatment for plaque erosion may prioritize antithrombotic therapy over PCI [10,65]. Potential benefits include avoidance of stent thrombus and restenosis, and prolonged dual antiplatelet therapy. The EROSION study [10] is the first proof-of-concept study to demonstrate that the feasibility and safety of anti-thrombotic therapy without stenting in patients with ACS caused by plaque erosion. When OCT revealed plaque erosion with TIMI 3 flow and non-obstructive stenosis, stent implantation can be avoided. The study may potentially change the current “one-size-fits-all” stenting strategy for ACS patients, therefore avoiding the short-term and long-term complications related to stent. On this basis, conservative treatment for ACS caused by non-obstructive plaque erosion is recommended by recent EAPCI expert document on clinical use of intracoronary imaging [59]. The ongoing EROSION II and EROSION III studies (2021 Transcatheter Cardiovascular Therapeutics (TCT), Orlando, Florida) further expanded the population and explored differences of reperfusion therapy strategy and clinical prognosis between angiography and OCT-guided STEMI patients. The results will provide further evidence on establishing and improving individualized treatment strategies for ACS patients based on lesion characteristics.

There are several limitations of our work. The proposed CNN model is based on 2D images, which may be inferior to physicians who often rely on adjacent frame information for plaque erosion detection. An 3D extension of the algorithm should be explored in the future. While our AI algorithm was trained and evaluated on a diverse and clinically representative demographic, the subjects were not fully representative of a global population. Plaque erosion is multifactorial, with age, sex and lifestyle factors such as smoking and diet known to contribute to disease risk. Another limitation is that developing the AI algorithm required the manual labeling of thousands of intravascular OCT images, which is time-consuming and laborious, so further work can be done through semi-supervised or active learning so that physicians only need to label one of several images, effectively reducing the workload.

5. Conclusions

In summary, we for the first time proposed a fully automated AI method for the detection of plaque erosion in intravascular OCT images using deep learning. Our method can provide an objective basis for clinical diagnosis, which may help enhance the diagnostic accuracy for plaque erosion in vivo and develop individualized treatment strategies for optimal management of ACS patients.

Funding

National Natural Science Foundation of China (62075033, 62135002, 61921002, 82061130223); Sichuan Science and Technology Program (No. 2020YFS0076); the Fundamental Research Funds for the Central Universities; Newton Fund (NAF\R11\1015).

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. R. Vedanthan, B. Seligman, and V. Fuster, “Global perspective on acute coronary syndrome: a burden on the young and poor,” Circ. Res. 114(12), 1959–1975 (2014). [CrossRef]  

2. E. Falk, M. Nakano, J. F. Bentzon, A. V. Finn, and R. Virmani, “Update on acute coronary syndromes: the pathologists’ view,” Eur. Heart J. 34(10), 719–728 (2013). [CrossRef]  

3. R. Virmani, F. D. Kolodgie, A. P. Burke, A. Farb, and S. M. Schwartz, “Lessons from sudden coronary death: a comprehensive morphological classification scheme for atherosclerotic lesions,” Arterioscler., Thromb., Vasc. Biol. 20(5), 1262–1275 (2000). [CrossRef]  

4. R. Virmani, A. P. Burke, A. Farb, and F. D. Kolodgie, “Pathology of the vulnerable plaque,” J. Am. Coll. Cardiol. 47(8), C13–C18 (2006). [CrossRef]  

5. A. C. Van Der Wal, A. E. Becker, C. Van der Loos, and P. Das, “Site of intimal rupture or erosion of thrombosed coronary atherosclerotic plaques is characterized by an inflammatory process irrespective of the dominant plaque morphology,” Circulation 89(1), 36–44 (1994). [CrossRef]  

6. A. Farb, A. P. Burke, A. L. Tang, Y. Liang, P. Mannan, J. Smialek, and R. Virmani, “Coronary plaque erosion without rupture into a lipid core: a frequent cause of coronary thrombosis in sudden coronary death,” Circulation 93(7), 1354–1363 (1996). [CrossRef]  

7. E. Durand, A. Scoazec, A. Lafont, J. Boddaert, A. Al Hajzen, F. Addad, M. Mirshahi, M. Desnos, A. Tedgui, and Z. Mallat, “In vivo induction of endothelial apoptosis leads to vessel thrombosis and endothelial denudation: a clue to the understanding of the mechanisms of thrombotic plaque erosion,” Circulation 109(21), 2503–2506 (2004). [CrossRef]  

8. Y. Sato, K. Hatakeyama, A. Yamashita, K. Marutsuka, A. Sumiyoshi, and Y. Asada, “Proportion of fibrin and platelets differs in thrombi on ruptured and eroded coronary atherosclerotic plaques in humans,” Heart (London, U. K.) 91(4), 526–530 (2005). [CrossRef]  

9. R. A. Partida, P. Libby, F. Crea, and I.-K. Jang, “Plaque erosion: a new in vivo diagnosis and a potential major shift in the management of patients with acute coronary syndromes,” Eur. Heart J. 39(22), 2070–2076 (2018). [CrossRef]  

10. H. Jia, J. Dai, J. Hou, L. Xing, L. Ma, H. Liu, M. Xu, Y. Yao, S. Hu, E. Yamamoto, H. Lee, S. Zhang, B. Yu, and I.-K. Jang, “Effective anti-thrombotic therapy without stenting: intravascular optical coherence tomography-based management in plaque erosion (the EROSION study),” Eur. Heart J. 38, 792–800 (2016). [CrossRef]  

11. D. Huang, E. A. Swanson, C. P. Lin, J. S. Schuman, W. G. Stinson, W. Chang, M. R. Hee, T. Flotte, K. Gregory, C. A. Puliafito, and J. G. Fujimoto, “Optical coherence tomography,” Science 254(5035), 1178–1181 (1991). [CrossRef]  

12. H. Jia, T. Kubo, T. Akasaka, and B. Yu, “Optical coherence tomography guidance in management of acute coronary syndrome caused by plaque erosion,” Circ. J. 82(2), 302–308 (2018). [CrossRef]  

13. H. Jia, F. Abtahian, A. D. Aguirre, S. Lee, S. Chia, H. Lowe, K. Kato, T. Yonetsu, R. Vergallo, S. Hu, J. Tian, H. Lee, S.-J. Park, Y.-S. Jang, O. C. Raffel, K. Mizuno, S. Uemura, T. Itoh, T. Kakuta, S.-Y. Choi, H. L. Dauerman, A. Prasad, C. Toma, I. McNulty, S. Zhang, B. Yu, V. Fuster, J. Narula, R. Virmani, and I.-K. Jang, “In vivo diagnosis of plaque erosion and calcified nodule in patients with acute coronary syndrome by intravascular optical coherence tomography,” J. Am. Coll. Cardiol. 62(19), 1748–1758 (2013). [CrossRef]  

14. J. Dai, C. Fang, S. Zhang, J. Hou, L. Xing, L. Li, Y. Wang, J. Wang, Y. Wang, Y. Tu, X. Zhang, H. Liu, M. Xu, X. Ren, L. Ma, H. Yu, G. Wei, S. Zhang, G. S. Mintz, and B. Yu, “Not All plaque erosions are equal: Novel insights from 1,660 patients with STEMI: A clinical, angiographic, and intravascular OCT Study,” JACC: Cardiovascular Imaging 13, 516–518 (2020). [CrossRef]  

15. G. J. Tearney, E. Regar, T. Akasaka, T. Adriaenssens, P. Barlis, H. G. Bezerra, B. Bouma, N. Bruining, J.-m. Cho, S. Chowdhary, M. A Costa, R. d. Silva, J. Dijkstra, C. D. Mario, D. Dudek, E. Falk, M. D Feldman, P. Fitzgerald, H. M Garcia-Garcia, N. Gonzalo, J. F Granada, G. Guagliumi, N. R Holm, Y. Honda, F. Ikeno, M. Kawasaki, J. Kochman, L. Koltowski, T. Kubo, T. Kume, H.i Kyono, C. C. S. Lam, G. Lamouche, D. P Lee, M. B Leon, A. Maehara, O. Manfrini, G. S Mintz, K. Mizuno, M.-a. Morel, S. Nadkarni, H. Okura, H. Otake, A. Pietrasik, F. Prati, L. Räber, M. D Radu, J. Rieber, M. Riga, A. Rollins, M. Rosenberg, V. Sirbu, P. W J C Serruys, K. Shimada, T. Shinke, J. Shite, E. Siegel, S. Sonoda, M. Suter, S. Takarada, A. Tanaka, M. Terashima, T. Thim, S. Uemura, G. J Ughi, H.M M v. Beusekom, A. F W van der Steen, G.-A. v. Es, G. v. Soest, R. Virmani, S. Waxman, N. J Weissman, and G. Weisz, “Consensus standards for acquisition, measurement, and reporting of intravascular optical coherence tomography studies: a report from the International Working Group for Intravascular Optical Coherence Tomography Standardization and Validation,” J. Am. Coll. Cardiol. 59(12), 1058–1072 (2012). [CrossRef]  

16. K. B. Obaid, S. Zeebaree, and O. M. Ahmed, “Deep learning models based on image classification: a review,” International Journal of Science and Business 4, 75–81 (2020). [CrossRef]  

17. V. Badrinarayanan, A. Kendall, and R. Cipolla, “Segnet: A deep convolutional encoder-decoder architecture for image segmentation,” IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017). [CrossRef]  

18. J. Jiang, H. M. Kasem, and K.-W. Hung, “Robust image completion via deep feature transformations,” IEEE Access 7, 113916–113930 (2019). [CrossRef]  

19. G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. Van Der Laak, B. Van Ginneken, and C. I. Sánchez, “A survey on deep learning in medical image analysis,” Med. Image Anal. 42, 60–88 (2017). [CrossRef]  

20. A. Esteva, B. Kuprel, R. A. Novoa, J. Ko, S. M. Swetter, H. M. Blau, and S. Thrun, “Dermatologist-level classification of skin cancer with deep neural networks,” Nature 542(7639), 115–118 (2017). [CrossRef]  

21. S. Pereira, A. Pinto, V. Alves, and C. A. Silva, “Brain tumor segmentation using convolutional neural networks in MRI images,” IEEE Trans. Med. Imaging 35(5), 1240–1251 (2016). [CrossRef]  

22. C. Huang, J. Wang, Q. Xie, and Y.-D. Zhang, “Analysis Methods of Coronary Artery Intravascular Images: A Review,” Neurocomputing 489, 27–39 (2022). [CrossRef]  

23. F. Tian, W. Yu, J. Huang, T. Zhang, Y. Chen, and S. Tu, “First presentation of integration of intravascular optical coherence tomography and computational fractional flow reserve,” Int J Cardiovasc Imaging 35(4), 601–602 (2019). [CrossRef]  

24. M. Miyagawa, M. G. F. Costa, M. A. Gutierrez, J. P. G. F. Costa, and C. F. Costa Filho, “Lumen segmentation in optical coherence tomography images using convolutional neural network,” in 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), (IEEE, 2018), 600–603.

25. C. Huang, Y. Lan, G. Xu, X. Zhai, J. Wu, F. Lin, N. Zeng, Q. Hong, E. Ng, Y. Peng, F. Chen, and G. Zhang, “A deep segmentation network of multi-scale feature fusion based on attention mechanism for IVOCT lumen contour,” IEEE/ACM Trans. Comput. Biol. and Bioinf. 18(1), 62–69 (2021). [CrossRef]  

26. C. Li, H. Jia, J. Tian, C. He, F. Lu, K. Li, Y. Gong, S. Hu, B. Yu, and Z. Wang, “Comprehensive assessment of coronary calcification in intravascular OCT using a spatial-temporal encoder-decoder network,” IEEE Trans. Med. Imaging 41(4), 857–868 (2022). [CrossRef]  

27. V. Baruah, A. Zahedivash, T. Hoyt, A. McElroy, D. Vela, L. M. Buja, A. Cabe, M. Oglesby, A. Estrada, P. Antonik, T. E. Milner, and M. D. Feldman, “Automated Coronary Plaque Characterization With Intravascular Optical Coherence Tomography and Smart-Algorithm Approach: Virtual Histology OCT,” JACC: Cardiovascular Imaging 13(8), 1848–1850 (2020). [CrossRef]  

28. X. Cao, J. Zheng, Z. Liu, P. Jiang, D. Gao, and R. Ma, “Improved U-Net for Plaque Segmentation of Intracoronary Optical Coherence Tomography Images,” in International Conference on Artificial Neural Networks, (Springer, 2021), 598–609.

29. J. Lee, Y. Gharaibeh, C. Kolluru, V. N. Zimin, L. A. P. Dallan, J. N. Kim, H. G. Bezerra, and D. L. Wilson, “Segmentation of Coronary Calcified Plaque in Intravascular OCT Images Using a Two-Step Deep Learning Approach,” IEEE Access 8, 225581–225593 (2020). [CrossRef]  

30. M. Chu, H. Jia, J. L. Gutiérrez-Chico, A. Maehara, Z. A. Ali, X. Zeng, L. He, C. Zhao, M. Matsumura, P. Wu, M. Zeng, T. Kubo, B. Xu, L. Chen, B. Yu, G. S. Mintz, W. Wijns, N. R. Holm, and S. Tu, “Artificial intelligence and optical coherence tomography for the automatic characterisation of human atherosclerotic plaques,” EuroIntervention 17(1), 41–50 (2021). [CrossRef]  

31. Y. Guo, L. Bi, A. Kumar, Y. Gao, R. Zhang, D. Feng, Q. Wang, and J. Kim, “Deep local-global refinement network for stent analysis in ivoct images,” in International Conference on Medical Image Computing and Computer-Assisted Intervention, (Springer, 2019), 539–546.

32. G. Yang, E. Mehanna, C. Li, H. Zhu, C. He, F. Lu, K. Zhao, Y. Gong, and Z. Wang, “Stent detection with very thick tissue coverage in intravascular OCT,” Biomed. Opt. Express 12(12), 7500–7516 (2021). [CrossRef]  

33. D. Ding, W. Yu, H. Tauzin, G. L. De Maria, P. Wu, F. Yang, R. A. Kotronias, D. Terentes-Printzios, M. Wolfrum, A. P. Banning, N. Meneveau, W. Wijns, and S. Tu, “Optical Flow Ratio for Assessing Stenting Result and Physiological Significance of Residual Disease,” EuroIntervention 17(12), e989–e998 (2021). [CrossRef]  

34. Z. Wang, H. Jia, J. Tian, T. Soeda, R. Vergallo, Y. Minami, H. Lee, A. Aguirre, J. G. Fujimoto, and I.-K. Jang, “Computer-aided image analysis algorithm to enhance in vivo diagnosis of plaque erosion by intravascular optical coherence tomography,” Circ: Cardiovascular Imaging 7(5), 805–810 (2014). [CrossRef]  

35. F. Lateef and Y. Ruichek, “Survey on semantic segmentation using deep learning techniques,” Neurocomputing 338, 321–348 (2019). [CrossRef]  

36. Y. Liu, P. Zhang, Q. Song, A. Li, P. Zhang, and Z. Gui, “Automatic segmentation of cervical nuclei based on deep learning and a conditional random field,” IEEE Access 6, 53709–53721 (2018). [CrossRef]  

37. C. Cortes and V. Vapnik, “Support-vector networks,” Mach Learn 20(3), 273–297 (1995). [CrossRef]  

38. R. Geirhos, J.-H. Jacobsen, C. Michaelis, R. Zemel, W. Brendel, M. Bethge, and F. A. Wichmann, “Shortcut learning in deep neural networks,” Nat Mach Intell 2(11), 665–673 (2020). [CrossRef]  

39. O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in International Conference on Medical image computing and computer-assisted intervention, (Springer, 2015), 234–241.

40. S. Hermann and R. Klette, “Multigrid analysis of curvature estimators,” (CITR, The University of Auckland, New Zealand, 2003).

41. S. Hermann and R. Klette, “A comparative study on 2d curvature estimators,” in 2007 International Conference on Computing: Theory and Applications (ICCTA'07), (IEEE, 2007), 584–589.

42. J. Sklansky, “Measuring concavity on a rectangular mosaic,” IEEE Trans. Comput. C-21(12), 1355–1364 (1972). [CrossRef]  

43. Z.-Q. Zhao, P. Zheng, S.-t. Xu, and X. Wu, “Object detection with deep learning: A review,” IEEE Trans. Neural Netw. Learning Syst. 30(11), 3212–3232 (2019). [CrossRef]  

44. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, and F. Li, “Imagenet large scale visual recognition challenge,” Int J Comput Vis 115(3), 211–252 (2015). [CrossRef]  

45. G. J. Tearney, H. Yabushita, S. L. Houser, H. T. Aretz, I.-K. Jang, K. H. Schlendorf, C. R. Kauffman, M. Shishkov, E. F. Halpern, and B. E. Bouma, “Quantification of macrophage content in atherosclerotic plaques by optical coherence tomography,” Circulation 107(1), 113–119 (2003). [CrossRef]  

46. C. Xu, J. M. Schmitt, S. G. Carlier, and R. Virmani, “Characterization of atherosclerosis plaques by measuring both backscattering and attenuation coefficients in optical coherence tomography,” J. Biomed. Opt. 13(3), 034003 (2008). [CrossRef]  

47. G. Van Soest, T. P. Goderie, E. Regar, S. Koljenovic, A. G. J. van Leenders, N. Gonzalo, S. van Noorden, T. Okamura, B. E. Bouma, G. J. Tearney, J. W. Oosterhuis, P. W. Serruys, and A. F. W. van der Steen, “Atherosclerotic tissue characterization in vivo by optical coherence tomography attenuation imaging,” J. Biomed. Opt. 15(1), 011105 (2010). [CrossRef]  

48. A. Rosenfeld, Digital picture processing (Academic press, 1976).

49. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in European conference on computer vision, (Springer, 2014), 740–755.

50. W. R. Crum, O. Camara, and D. L. Hill, “Generalized overlap measures for evaluation and validation in medical image analysis,” IEEE Trans. Med. Imaging 25(11), 1451–1461 (2006). [CrossRef]  

51. M. Hossin and M. N. Sulaiman, “A review on evaluation metrics for data classification evaluations,” IJDKP 5, 01–11 (2015). [CrossRef]  

52. S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017). [CrossRef]  

53. X. Chen, X. Wang, K. Zhang, K.-M. Fung, T. C. Thai, K. Moore, R. S. Mannel, H. Liu, B. Zheng, and Y. Qiu, “Recent advances and clinical applications of deep learning in medical image analysis,” Med. Image Anal. 79, 102444 (2022). [CrossRef]  

54. F. Isensee, P. F. Jaeger, S. A. Kohl, J. Petersen, and K. H. Maier-Hein, “nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation,” Nat. Methods 18(2), 203–211 (2021). [CrossRef]  

55. F. Milletari, N. Navab, and S.-A. Ahmadi, “V-net: Fully convolutional neural networks for volumetric medical image segmentation,” in 2016 fourth international conference on 3D vision (3DV), (IEEE, 2016), 565–571.

56. X. Wang, R. Zhang, T. Kong, L. Li, and C. Shen, “Solov2: Dynamic and fast instance segmentation,” Advances in Neural information processing systems 33, 17721–17732 (2020). [CrossRef]  

57. S. Kitahara, Y. Kataoka, H. Sugane, F. Otsuka, Y. Asaumi, T. Noguchi, and S. Yasuda, “In vivo imaging of vulnerable plaque with intravascular modalities: Its advantages and limitations,” Cardiovasc Diagn Ther 10(5), 1461–1479 (2020). [CrossRef]  

58. Y. Ozaki, M. Okumura, T. F. Ismail, S. Motoyama, H. Naruse, K. Hattori, H. Kawai, M. Sarai, Y. Takagi, J. Ishii, H. Anno, R. Virmani, P. W. Serruys, and J. Narula, “Coronary CT angiographic characteristics of culprit lesions in acute coronary syndromes not related to plaque rupture as defined by optical coherence tomography and angioscopy,” Eur. Heart J. 32(22), 2814–2823 (2011). [CrossRef]  

59. T. W. Johnson, L. Räber, C. Di Mario, C. Bourantas, H. Jia, A. Mattesini, N. Gonzalo, J. M. De La Torre Hernandez, F. Prati, K. Koskinas, M. Joner, M. D. Radu, D. Erlinge, E. Regar, V. Kunadian, A. Maehara, R. A. Byrne, D. Capodanno, T. Akasaka, W. Wijns, G. S. Mintz, and G. Guagliumi, “Clinical use of intracoronary imaging. Part 2: Acute coronary syndromes, ambiguous coronary angiography findings, and guiding interventional decision-making: An expert consensus document of the European Association of Percutaneous Cardiovascular Interventions: Endorsed by the Chinese Society of Cardiology, the Hong Kong Society of Transcatheter Endocardiovascular Therapeutics (HKSTENT) and the Cardiac Society of Australia and New Zealand,” Eur. Heart J. 40(31), 2566–2584 (2019). [CrossRef]  

60. H. Jinnouchi, R. Virmani, and A. V. Finn, “Are characteristics of plaque erosion defined by optical coherence tomography similar to true erosion in pathology?” Eur. Heart J. 39(22), 2086–2089 (2018). [CrossRef]  

61. A. C. Fahed and I.-K. Jang, “Plaque erosion and acute coronary syndromes: phenotype, molecular characteristics and future directions,” Nat. Rev. Cardiol. 18(10), 724–734 (2021). [CrossRef]  

62. E. Yamamoto, T. Yonetsu, T. Kakuta, T. Soeda, Y. Saito, B. P. Yan, O. Kurihara, M. Takano, G. Niccoli, T. Higuma, S. Kimura, Y. Minami, J. Ako, T. Adriaenssens, N. F. Boeder, H. M. Nef, F. Fracassi, T. Sugiyama, H. Lee, F. Crea, T. Kimura, J. G. Fujimoto, V. Fuster, and I.-K. Jang, “Clinical and laboratory predictors for plaque erosion in patients with acute coronary syndromes,” J. Am. Heart Assoc. 8(21), e012322 (2019). [CrossRef]  

63. F.-J. Neumann, M. Sousa-Uva, A. Ahlsson, F. Alfonso, A. P. Banning, U. Benedetto, R. A. Byrne, J.-P. Collet, V. Falk, S. J. Head, P. Jüni, A. Kastrati, A. Koller, S. D. Kristensen, J. Niebauer, D. J. Richter, P. M. Seferovic, D. Sibbing, G. G. Stefanini, S. Windecker, R. Yadav, and M. O. Zembala, “2018 ESC/EACTS Guidelines on myocardial revascularization,” Eur. Heart J. 40(2), 87–165 (2019). [CrossRef]  

64. S. Cook, P. Eshtehardi, B. Kalesan, L. Räber, P. Wenaweser, M. Togni, A. Moschovitis, R. Vogel, C. Seiler, F. R. Eberli, T. Lüscher, B. Meier, P. Jüni, and S. Windecker, “Impact of incomplete stent apposition on long-term clinical outcome after drug-eluting stent implantation,” Eur. Heart J. 33(11), 1334–1343 (2012). [CrossRef]  

65. R. Vergallo, I.-K. Jang, and F. Crea, “New prediction tools and treatment for ACS patients with plaque erosion,” Atherosclerosis 318, 45–51 (2021). [CrossRef]  

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (9)

Fig. 1.
Fig. 1. Plaque erosion and typical atherosclerotic plaques on intravascular OCT images. (a) Plaque erosion. (b) Fibrous plaque from the same vessel 9.8 mm proximal from the erosion site in Fig. 1(a). (c) Lipid plaque. (d) Calcified plaque.
Fig. 2.
Fig. 2. Artificial intelligence (AI) based plaque erosion detection algorithm. The original image and two key shape features convexity and curvature are input into a shape-encoded convolutional neural network. The preliminary detected mask is fed to the second module, where post-processing based on domain knowledge is performed to generate the final output.
Fig. 3.
Fig. 3. Characterization of convexity and curvature of plaque erosion. From left to right are the original image, enlarged view of the lumen, convexity and curvature. Convexity: computed as the difference between the convex hull of the lumen and the lumen itself, labeled in cyan. Curvature: defined as the rate of change of slope of the lumen boundary curve, labeled in a continuous colormap.
Fig. 4.
Fig. 4. Shape-encoded CNN model for plaque erosion detection in intravascular OCT. The original image, convexity and curvature are fed into feature extraction modules to generate ROIs via Region Proposal Network (RPN). The RCNN Head and Mask Head output the detection results.
Fig. 5.
Fig. 5. Schematic diagram of the classification model using clinically interpretable features. The area corresponding to the mask output from the shape-encoded convolutional neural network in the original image is used to extract clinically interpretable features, which are input into a SVM model to obtain the classification results of plaque erosion.
Fig. 6.
Fig. 6. Plaque erosion detection by the proposed AI algorithm in four cases, with each column showing one case. From top to bottom are the original images, ground truth, and predicted plaque erosion by our proposed method respectively.
Fig. 7.
Fig. 7. Precision-recall curve (orange) shows the performance of the automated prediction of plaque erosion against ground truth. Operating point A (73% precision) and B (95% recall) are indicated by the blue and green triangles respectively. Blue-shaded areas indicate 95% confidence intervals.
Fig. 8.
Fig. 8. Undetected plaque erosion, with each column showing one case. From top to bottom are the original images and ground truth respectively. The blue circle, green triangle, and pink rhombus on the top right respectively indicates physician 1, physician 2 and physician 3 agreed on the case.
Fig. 9.
Fig. 9. Likely probable OCT-plaque erosion detected by the AI algorithm but missed by physicians, with each column showing one case. From top to bottom are the original images and the output from the algorithm respectively.

Tables (7)

Tables Icon

Table 1. Comparison of different shape encoder modules

Tables Icon

Table 2. List of the clinically interpretable features

Tables Icon

Table 3. Quantitative comparison with state-of-the-art CNN methods

Tables Icon

Table 4. Quantitative comparison between the proposed AI algorithm and three physicians

Tables Icon

Table 5. Quantitative comparison on Mask RCNN with convexity with different encoder modules

Tables Icon

Table 6. Quantitative comparison on Mask RCNN with convexity with different sizes of feature map

Tables Icon

Table 7. Quantitative comparison on algorithms with different combinations of shape features

Equations (1)

Equations on this page are rendered with MathJax. Learn more.

P r e c i s i o n = T P T P + F P R e c a l l = T P T P + F N .
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.