1.

Introduction

Coronary artery disease (CAD) remains the leading cause of death worldwide.^1,2 It is caused by plaque deposition in one or multiple coronary arteries producing a narrowing of the lumen called stenosis. Obstructive stenoses restrict the coronary blood supply, which can inhibit normal myocardial function and cause ischemia. Possible treatment options for patients with suspected stable CAD include lifestyle change, medical therapy, percutaneous coronary intervention (PCI), and coronary artery bypass grafting (CABG).³ PCI and CABG are invasive revascularization procedures and expose patients to multiple risks, including vascular damage, contrast-induced renal failure, myocardial infarction, and sepsis.⁴

Many morphologically apparent stenoses are not functionally significant. Due to the risks associated with invasive treatment procedures, treatment of functionally insignificant stenoses should be avoided. In the FAME study, disregarding measurements of fractional flow reserve during PCI treatment increased the risk of death or myocardial infarction within 2 years by 50%.⁵ Therefore, the choice of treatment strategy is frequently based on both morphological and functional information.^3,5,6–8

Morphological information can be noninvasively obtained from coronary computed tomography angiography (CCTA) as high-resolution three-dimensional (3-D) images of the coronary artery tree and stenoses.⁷ However, due in part to low temporal resolution, CCTA is a poor predictor of the functional significance of stenoses.^7,9 Functional information can be noninvasively obtained from stress three-dimensional echocardiography (3DE) in which stress, induced by exercise or pharmacological agents, increases the flow demand in stenotic coronary arteries. The increased demand can induce ischemia, resulting in wall motion abnormalities that can be directly observed.⁸ Other noninvasive functional imaging modalities include single-photon emission computed tomography (SPECT),¹⁰ positron emission tomography (PET),¹¹ and cardiac magnetic resonance imaging.¹² Advantages of 3DE include low cost and lack of ionising radiation. The current gold standard for functional imaging in CAD is invasive coronary angiography,⁸ which carries greater risks of death and myocardial infarction than noninvasive alternatives.¹³ Furthermore, only one-third of coronary angiography investigations are followed up by revascularization procedures,¹³ suggesting that improved noninvasive imaging modalities could reduce patient exposure to catheterization and associated complications.

To allocate stenotic segments to functional defects, morphological and functional information is traditionally viewed side-by-side and integrated mentally, a time-consuming and error-prone process. Side-by-side interpretation is further complicated by large individual variation of the coronary artery tree anatomy and myocardial distribution territories.^14,15 These difficulties can be alleviated by fusing morphological and functional information into a single image with a single spatial frame of reference. In a study comparing fused and side-by-side interpretations of CCTA and SPECT myocardial perfusion images, image fusion enabled confident assessment of the functional significance of most stenoses that could not be classified in images viewed side-by-side.¹⁶ Therefore, cardiac image fusion might reduce unnecessary exposure to invasive diagnostic procedures, ultimately reducing mortality in CAD patients.

Image fusion of CCTA and SPECT myocardial perfusion images enables more confident assessment of the functional significance of most stenoses than viewing images side-by-side.¹⁶ Therefore, cardiac image fusion might facilitate interpretation and reduce unnecessary exposure to invasive diagnostic procedures, ultimately reducing mortality in CAD patients.

Fusion of CCTA and 3DE has recently been shown to be feasible.^17,18 In this paper, we describe three methods for automatic CCTA/3DE registration based on segmentation and alignment. These methods are validated and evaluated in patients with suspected CAD by comparison with manually identified landmarks. Note that this paper represents an extension of preliminary results published in conference proceedings.¹⁹

2.

Methods

2.4.

Image Registration

The 3DE volume was registered to the CCTA volume using three alternative methods: landmark distance minimization based on Procrustes alignment,²¹ endocardium iterative closest point (ICP) alignment,²² and three-chamber alignment. Each registration method estimated a rigid transformation based on segmented anatomical landmarks in the CCTA volume and the 3DE volume of matching cardiac phase, which was 75% of the RR interval in nine patients and 35% in two.

The segmented landmarks were mitral valve centers $M_{\mathrm{CCTA}}$ and $M_{3\mathrm{DE}}$, outflow tract centers $O_{\mathrm{CCTA}}$ and $O_{3\mathrm{DE}}$, and endocardial apices $A_{\mathrm{CCTA}}$ and $A_{3\mathrm{DE}}$. Furthermore, the endocardial surfaces were segmented and represented as point clouds $\{P_{\mathrm{CCTA},i}\}$ and $\{P_{3\mathrm{DE},i}\}$, $i=1,\dots ,642$.

Landmark distance minimization: The mean squared distances between corresponding landmarks from the two volumes was minimized, i.e., rigid Procrustes analysis without scaling or reflection.²¹ Landmarks used were endocardial apex, mitral valve center, and LVOT. As such, the rigid transformation used was the matrix $T_{\mathrm{proc}}$ that minimized the sum

$${|A_{\mathrm{CCTA}}-T_{\mathrm{proc}}{A}_{3\mathrm{DE}}|}^2+{|M_{\mathrm{CCTA}}-T_{\mathrm{proc}}{M}_{3\mathrm{DE}}|}^2+{|O_{\mathrm{CCTA}}-T_{\mathrm{proc}}{O}_{3\mathrm{DE}}|}^2.$$

Endocardium ICP alignment: The segmented endocardial surfaces from CCTA and 3DE, 642 points each, were aligned using the ICP method: the two surfaces were initially registered using the landmark distance minimization described above. Each point on the 3DE surface was associated to the closest point on the CCTA surface. A new transform was obtained by minimizing the mean squared distance of these 642 point pairs. Points were reassociated and a new transform obtained until the process converged.

Three-chamber alignment: The three-chamber views were aligned by the unique rigid transformation such that

• 1. Mitral valve centers of both volumes coincided, i.e., $M_{\mathrm{CCTA}}=T_{3\mathrm{ch}}{M}_{3\mathrm{DE}}$.

• 2. The long axes, as defined from mitral valve center to endocardial apex, coincided, i.e., the points $A_{\mathrm{CCTA}}$, $M_{\mathrm{CCTA}}$, $T_{3\mathrm{ch}}{A}_{\mathrm{CCTA}}$, and $T_{3\mathrm{ch}}{M}_{\mathrm{CCTA}}$ were collinear.

• 3. The three-chamber planes, as defined by mitral valve centers, apices and LVOT, coincided, i.e., the points $A_{\mathrm{CCTA}}$, $M_{\mathrm{CCTA}}$, $O_{\mathrm{CCTA}}$, $T_{3\mathrm{ch}}{A}_{\mathrm{CCTA}}$, $T_{3\mathrm{ch}}{M}_{\mathrm{CCTA}}$, and $T_{3\mathrm{ch}}{O}_{\mathrm{CCTA}}$ were coplanar.

Registration with the described methods required no manual intervention, other than the semiautomatic segmentation. All methods were implemented in C++ using the Visualization Toolkit.²³ Figure 1 shows schematic diagrams of the three registration methods.

Fig. 1

Schematic diagram of LVs from CCTA (black) and 3DE (red), registered using different registration methods: (a) landmark distance minimization, which minimizes the squared distances between apices (A), mitral valve centers (MVC), and left-ventricular outflow tracts (LVOT). (b) Endocardium ICP alignment, which minimizes distances between endocardial surfaces (inner wavy lines). (c) Three-chamber alignment, which aligns the mitral valve centers (MVC), long axes (A-MVC), and three-chamber planes (A-MVC-LVOT). (d) Demonstrates the landmarks overlaid on an echocardiographic three-chamber view.

2.5.

Anatomical Validation

Anatomical accuracy of the registration methods was quantitatively evaluated based on anatomical landmarks present in both volumes. An experienced cardiologist used a custom multiplanar reformatting visualization software to identify these LV anatomical landmarks. Note that the landmarks identified through this validation procedure are marked manually and are distinct from those detected in the semiautomatic segmentation step.

The procedure performed for each volume was as follows:

The software presented three approximate long-axis and one short-axis view simultaneously. The user adjusted these views to obtain an optimally aligned three-chamber view. Figures 2(a) and 2(b) show the adjusted views from one 3DE and one CCTA volume. The software internally produced additional views by rotating the identified three-chamber view around the normal of the short-axis view. The user was presented with the three-chamber view rotated by 0 deg, 60 deg, and 120 deg consecutively. In each view, the user marked the endocardial apex. The user was presented with the three-chamber view and marked the center of the aortic valve. The user was presented with the three-chamber view rotated by 0 deg through 165 deg in 15-deg increments consecutively. In each view, the user marked the two points of the mitral annulus. Figures 2(c)–2(d) show the 60-deg view and identified points for one patient. Finally, the user was again presented with the three-chamber view rotated by 0 deg, 60 deg, and 120 deg consecutively. In each view, the user delineated the endocardium.

Fig. 2

Screen captures from the custom software used to identify landmarks for evaluation of anatomical accuracy. (a) The interface used to identify standard views in a CCTA volume. (c) The three-chamber view rotated by 60 deg, which was one of the planes used to identify the mitral annulus. (b) and (d) The corresponding images from a 3DE volume.

The aforementioned landmark identification workflow was performed first for each 3DE volume in a random order, then for each CCTA volume in a random order. For the 4-D volumes from 3DE, the cardiologist viewed the entire cine loop but marked the landmarks only in the time frame of cardiac phase matching the CCTA volume. The LV endocardial apex in a volume was taken as the mean of the three apical landmarks identified in different views and the mitral annular center position as the mean of the 24 mitral annulus landmarks.

Anatomical accuracy was quantified in each fused volume as the intermodality distances between corresponding anatomical landmarks [Fig. 3(a)]. Apex distance was defined as the distance from endocardial apex in the CCTA volume to endocardial apex in the 3DE volume. Mitral annular center distance and aortic valve center distance were defined correspondingly.

Fig. 3

Schematic diagram of metrics used to quantify registration quality: (a) distances between mitral centers, aortic centers, and apices, (b) transverse-plane angle, and (c) long-axis angle.

Furthermore, two intermodality angles were calculated to quantify LV rotation: long-axis angle and transverse-plane angle [Figs. 3(b)–3(c)]. The long axis in each modality was estimated as the line from endocardial apex to the mitral annular center. The long-axis angle was defined as the angle between the long axes from the two modalities. To estimate the transverse-plane angle, an average was first formed of the two long axes from 3DE and CCTA. The transverse-plane angle was defined as the angle between the aortic valve centers with respect to the average long axis.

Finally, two additional measures were used to assess endocardial surface similarity: the mean point-to-point distance was obtained by averaging the distances from each point on the segmented 3DE endocardial surface to the CCTA surface. Dice’s coefficient was obtained by considering the intersection between the three-chamber view, as identified by a cardiologist in the CCTA volume using the visualization software described above and the two segmented endocardial surfaces. The two intersections, the 3DE three-chamber LV area $A$ and the CCTA three-chamber LV area $B$, were combined as follows to obtain Dice’s coefficient in the three-chamber view:

$$\mathrm{Dice}=2\frac{|A\cap B|}{|A|+|B|}.$$

The Wilcoxon signed-rank test was used to assess whether the quality measures vary between the registration methods presented. Tests were performed for each quality measure between each pair of registration methods for a total of $7\times 3=21$ tests.

3.

Results

Table 1 summarizes the results of the chosen quality metrics for the three registration methods applied on datasets from 11 patients. Landmark distance magnitude is around 1 to 2 cm for all registration methods. Rotational measures are similar for landmark distance minimization and three-chamber alignment, but worse for ICP. ICP performs better with regard to mean point-to-point distance. The three-chamber Dice’s coefficient is similar among all methods.

Table 1

Quality measures for three registration methods (n=11).

Figures 4(a)–4(c) give a qualitative view of the performance of the three evaluated registration methods for four representative patients by showing cross sections of the segmented endocardial surfaces from CCTA and 3DE, registered using the different methods. Figures 5(d) and 5(e) show cross sections of the semiautomatically segmented endocardial surfaces, used for registration, along with the manually marked endocardial surfaces, to give a qualitative view of the performance of the segmentation processes. Figure 5(f) overlays slices from CCTA with the corresponding slices from 3DE, as estimated using three-chamber alignment.

Fig. 4

Three-chamber cross-sections of the segmented LV surfaces from CCTA (cyan) and 3DE (magenta), registered with (a) three-chamber alignment, (b) landmark distance minimization, and (c) LV surface ICP. Each column corresponds to one of four representative patients.

Fig. 5

Row (a) shows three-chamber cross sections of CCTA images with semiautomatically segmented LV surfaces (red) and manually marked LV surfaces (green). Row (b) shows three-chamber cross sections of 3DE images with segmented (red) and manually marked (green) LV surfaces. Row (c) shows three-chamber cross sections of CCTA images with the same slice of the corresponding 3DE images overlaid. For row (c), images were registered using three-chamber alignment. Columns correspond to the columns of Fig. 4.

Statistical analysis on the quality measures presented in Table 1 was performed for each measure and each pair of registration methods for the entire population ($n=11$). Significant ($p<0.05$) difference was observed in six cases: between three-chamber alignment and landmark distance minimization with respect to apex distance ($p=0.002$), between landmark distance minimization and ICP with respect to aortic center distance ($p=0.042$), transverse-plane angle ($p=0.014$) and mean point-to-point distance ($p=0.007$), and between three-chamber alignment and ICP with respect to transverse-plane angle ($p=0.010$) and mean point-to-point distance ($p=0.002$).

Table 2 summarizes the inter- and intraobserver variability. The differences in landmark distance are on the order of 1 cm and smaller than the intermodality landmark distances as presented in Table 1. The inter- and intraobserver errors are similar.

Table 2

Intra- and interobserver variability. Segmentation was repeated for n=4 patients, once with the original operator and once with another operator. Repeated registration was performed with three-chamber alignment. Variability was quantified using differences in landmark positions and angles between original and repeated segmentations.

Figure 6 shows a fused image of anatomical information, the coronary artery tree from CCTA, and functional information, strain from 3DE, belonging to one of the included patients.

Fig. 6

Fused image of the coronary artery tree from CCTA and the LV endocardium from 3DE in end-diastole. Instantaneous longitudinal strain is color-coded on the LV surface.

4.

Discussion

This study aimed at comparing and validating three methods for automatic CCTA/3DE registration: landmark distance minimization, endocardium ICP alignment, and alignment of three-chamber views. All three registration methods showed good performance with regard to landmark distances, as seen in Table 1, with no obvious differences between the methods. Significant differences were found only between landmark distance minimization and three-chamber alignment with regard to apex distance ($p=0.002$) and between landmark distance minimization and ICP with regard to aortic center distance ($p=0.042$). Landmark distance minimization and three-chamber alignment performed better than ICP alignment with regard to the transverse-plane angle ($p=0.014$ and $p=0.010$, respectively). The anatomical landmarks used in landmark distance minimization and three-chamber alignment provide for more accurate rotational alignment, compared with ICP, which is likely the reason why the ICP method performed worse. The rotational component is crucial to properly relating LV segments and arteries. Furthermore, this result suggests that there was a difference in LV endocardial shape between CCTA and 3DE, caused by differences in actual shape or in segmented shape. Endocardium ICP alignment performed slightly better with regard to mean point-to-point distance, which is very similar to the metric minimized by the method ($p=0.002$ and $p=0.007$ when compared with three-chamber alignment and landmark distance minimization, respectively). This further indicates that a significant shape difference exists. Three-chamber Dice’s coefficient varied very little among methods.

Differences in actual shape are unavoidable due to the difference in the nature of CCTA and 3DE. In CCTA, the patient was medicated to decrease the heart rate, which likely reduced LV volume throughout the heart cycle,²⁴ whereas no medication was given during 3DE examination. The 3DE and CCTA volumes were registered at the same temporal phase as measured in fraction of the R-R interval. However, changes in heart rate mostly affect the length of the diastolic phase,²⁵ which may have resulted in a fusion of data from slightly different cardiac phases. The patient was also positioned differently during the two imaging examinations, which may be another reason for differences in LV volume and shape. 3DE was performed in the left lateral decubitus position, whereas the patient was positioned in the supine position during CCTA. Differences in respiratory motion may also have contributed to differences in LV shape and size. Due in part to these anatomical differences, perfect alignment of the two volumes would be impossible. Using an affine ICP algorithm²⁶ would possibly result in better alignment of the LV shapes and thereby reduce registration errors. However, better alignment of LV shapes would not necessarily result in an improved correlation of stenotic arteries with functional abnormalities. The weak performance of the ICP method is more likely due to differences in LV orientation, which would not be improved using an affine ICP algorithm.

Cardiac image fusion systems combining CCTA with PET or SPECT have been shown to contribute to facilitated decision on revascularization strategy compared with side-by-side analysis¹⁶ and reduction of the frequency of subsequent revascularization procedures.²⁷ Fusion of CCTA and 3DE has recently been shown to be feasible,^17,18 but, to ascertain whether or not CCTA/3DE image fusion could be clinically viable and provide incremental value in CAD treatment decision, clinical trials must be performed. It is plausible that the registration accuracy shown here is adequate to correlate stenotic arteries with functional abnormalities in a clinical setting. The American Heart Association recommends dividing the LV into 17 segments when assessing myocardial function.²⁸ In this model, the LV is divided into three levels along the long axis of the heart (apex, mid, and base). The basal and mid parts are then divided into six segments of 60 deg each, whereas the apical part is divided in four segments of 90 deg each and the apical cap, resulting in segments of an approximate size of $2\mathrm{cm}\times 2\mathrm{cm}$. The segment size is thus larger than the registration errors of the proposed registration methods, which implies that the registration accuracy would likely be deemed acceptable for correlation of stenotic arteries with functional abnormalities in a clinical setting. Further, fusion of CCTA and SPECT has shown a registration accuracy of less than 10 mm²⁹ and the clinical value of a similar fusion system has been demonstrated by increased specificity and positive predictive value compared with CCTA alone.³⁰ Registration accuracy for CCTA/SPECT fusion is in the same magnitude but slightly smaller than the demonstrated registration accuracy in this study, which probably is related to larger differences in LV shape between CCTA and 3DE than between SPECT and CT. Still, these results indicate that the methods proposed in this study may also have potential for clinical use.

Patients with complex CAD and multivessel disease are frequently referred to revascularization procedures, although studies have shown that stenoses found on their angiograms often are functionally insignificant.³¹ Cardiac image fusion may be beneficial for this patient group by facilitating identification of the culprit lesion responsible for perfusion defect and avoiding invasive procedures with risk of adverse effects.

A benefit of using 3DE as functional modality in cardiac image fusion instead of SPECT or PET is the lack of additional ionizing radiation. Still, the individual radiation exposure needs to be minimized and the risk versus benefit of each imaging examination must be taken into account. Over the last years, CCTA has emerged as a promising noninvasive alternative to invasive coronary angiography due to improved spatial resolution and lower radiation dose. The conventional way of analyzing coronary artery anatomy is by curved multiplanar 2-D slices in a 3-D CCTA volume. 3-D techniques alone have demonstrated insufficient performance in the evaluation of stenosis³² and need further development or to be used in combination with 2-D techniques. In cardiac image fusion, 3-D visualization of the coronary artery tree has the advantage of allowing for easier spatial assignment of stenosis and dysfunctional myocardium. A possible future extension of the proposed fusion method is to include simulated FFR measurements from CCTA as a measure of hemodynamic relevance of a stenosis.³³

A limitation of this study is the small group of patients enrolled and only one female patient was included. This was partly due to lack of female patients with suspected CAD undergoing CCTA on clinical indication in our clinic. However, this study aimed at comparing and demonstrating the potential of different registration methods for automatic CCTA/3DE registration. For this purpose, the included patient group was considered to properly represent a variety of cardiac anatomies present in a patient population with known or suspected CAD. The clinical effectiveness of the proposed fusion technique still needs to be determined in larger studies of both female and male patient groups.

The registration methods presented depended on proper segmentation since the registration is based on alignment of either landmarks or surfaces determined by the segmentation. These segmentation algorithms have been validated in previous studies with credible results.^34,35 However, the registration methods are semiautomatic and hence user-dependent since manual initialization is required for the segmentation step. Therefore, the intra- and interobserver variability for repeated segmentations was investigated and found to be overall smaller than the corresponding registration errors. This variation was considered to be acceptable and unlikely to be significant in a clinical setting. Fully automatic segmentation algorithms,^36,37 which could be used to further eliminate the user dependency in the registration process, also exist. However, the proposed semiautomatic approach is more time-efficient and likely more robust compared with a manual registration process, with a total processing time of $\sim 20\min$ per patient. Optimization of user interfaces could further reduce processing time to about 5 min per patient.

Myocardial strain assessment has been shown to be useful in the detection of hemodynamically significant stenosis, in particular, when performing stress echocardiography.³⁸ The proposed fusion could be extended to include functional parameters from stress echocardiography by adding registration of baseline and stress echocardiography. In addition to myocardial strain assessment, stress echocardiography enables real-time quantification of myocardial perfusion, which has shown promising results in the evaluation of myocardial ischemia in different patient populations and settings.^39,40 However, the number of clinical studies presented is still relatively small and mostly limited to two-dimensional views of the myocardium. Further development is thus needed before presenting reliable volumetric perfusion data as a functional parameter in the proposed cardiac image fusion. The optimal parameter or combination of functional parameters from 3DE to be visualized in the fusion needs to be investigated in further studies.

Traditionally, fusion of morphological and functional information is performed by mental integration that relies on models allocating myocardial segments to specific coronary arteries.⁴¹ However, these models have been shown to poorly assign myocardial segments with coronary arteries because of a highly varying coronary anatomy among individuals.⁴² The 3-D visualization approach proposed in this study allows for improved assignment of arteries and corresponding myocardial segments. This could be further facilitated by automatic visualization of a coronary artery and its most probable perfused myocardial segment. The feasibility of such visualization has not been investigated in the present study.

The registration of CCTA/3DE data was performed in a single time frame, 35% or 75% of the R-R interval. The CCTA was performed in these time frames since the heart is moving at least at end systole (35% of R-R interval) and middiastole (75% of R-R interval) in patients with heart rates below 80 beats per minute.⁴³ A limitation of using CCTA data from a single time frame is that the coronary artery tree remains static while the functional parameter from 3DE is visualized with high temporal resolution. This can lead to additional spatial misalignment, since the coronary artery motion has been shown to vary with heart rate and throughout the cardiac cycle.⁴³ The use of multiphase CCTA in future versions of the fusion technique would thus most likely improve the alignment of coronary arteries and myocardial segments throughout the entire cardiac cycle.

5.

Conclusion

Three methods for registration of CCTA and 3DE volumes were implemented and compared on data from patients with suspected CAD. All methods provided adequate registration and performed well with regard to landmark distances. Landmark distance minimization and alignment of the three-chamber planes performed better with regard to LV rotation than minimization of distances between endocardial surfaces (ICP). However, clinical studies must be performed to ascertain whether or not CCTA/3DE image fusion could be clinically viable and provide incremental value in CAD treatment decision.