Direct measurement of atrioventricular valve regurgitant jets using 4D flow cardiovascular magnetic resonance is accurate and reliable for children with congenital heart disease: a retrospective cohort study

The magnetic field, intravenous contrast, and technical parameters were prescribed by radiologists with expertise in CMR as per routine clinical practice. CMR exams were performed on 1.5 T CMR system (General Electric Healthcare, Waukesha, Wisconsin, USA; Optima 450 W) and 3 T (General Electric Healthcare, GE MR750) scanners. At our institution, it is standard practice with 4DF CMR to use ferumoxytol (Feraheme; AMAG Pharmaceuticals, Waltham, Massachusetts, USA) for 3 T and gadolinium-based contrasts (Gadobenate dimeglumine [Multihance, Bracco Diagnostics, Monroe, New Jersey, USA]; Gadofosveset) for 1.5 T scans to optimize signal-to-noise ratio [41].

Two-dimensional CMR planimetry utilized short-axis, balanced steady-state free-precession (bSSFP) and PC sequences with retrospective gating. Short-axis slices were obtained with a breath hold using one signal average and parallel imaging acceleration; 12–14, 8 mm slices spanned the ventricles at 20 phases per cardiac cycle. Matrix and temporal resolution parameters were adjusted to accommodate breath holding capacity. In patients who could not breath hold, multiple signal averages were employed. 2D PC measurements of atrioventricular forward flow (AVFF) were obtained with a velocity encoding (VENC) of 150 m/s as default, then increased until no aliasing was noted. The following parameters were used: temporal resolution 36.4 msec (range 21–54), acquired spatial resolution 256 × 192, and slice thickness 10 mm. There was no adjustment for through-plane motion for 2D PC measurements of AVFF. 2D PC measurements of AVFF were available for only 13/44 (30%) patients, thus 4DF CMR AVFF measurements via a static plane were obtained as a substitute for the purposes of this study.

4DF acquisitions were obtained as the last sequence of each CMR study following administration of contrast in the majority of patients (Table 1). Non-contrast enhanced 4DF acquisition was obtained in one subject with a historical allergy to gadolinium-based contrast. 1.5 T CMR images were obtained with the following parameters, mean (range): flip angle of 15*, reconstructed spatial resolution of 1.2 (0.8–1.5) × 1.4 (0.8–1.8) × 2.1 (0.9–3) mm³, echo time 1.9 (1.5–2.1) msec, repetition time 4.4 (3.6–5.1) msec, views-per-segment of 3 (2–5) depending on heart rate, temporal resolution of 62.5 msec (28.6–83.6), bandwidth of ±63.3 kHz (62–125), and VENC parameters of 300 (250–450 m/sec) chosen to avoid aliasing. The base acceleration factor used was 5 × 3 with motion correction performed by leveraging the velocity encoding gradients as intrinsic navigators and soft-gating the data, for an approximate overall acceleration factor of 30. 3 T CMR images were obtained with the following parameters, mean (range): flip angle of 15*, reconstructed spatial resolution of 0.84 (0.7–1.2) × 0.9 (0.7–1.6) × 1.7 (1.2–2.8) mm³, TE 1.8 (1.7–2.1) msec, TR 4.1 (3.7–4.8) msec, true temporal resolution of 32.4 msec (18.3–58.6), bandwidth of ±83.3 kHz (63–100), total acceleration factor of 2.0 × 2.0, and VENC 300 (250–550 m/sec) chosen to avoid aliasing. Retrospective gating was used to obtain a total of 20 cardiac phases for 1.5 T and 20–30 phases for 3 T scanners. Due to clinical workflow, many of our 3 T exams are performed sedated or with general anesthesia and the vast majority of the exams performed on the 1.5 T magnet are performed unsedated. For this reason, there is a greater effort to accelerate sequences performed on our 1.5 T magnet.

To minimize coherent artifacts from respiratory motion, a pseudo-random view-ordering and sampling was used for 4DF [42]. More specifically, a variable-density Poisson disc sampling mask was first generated [43]. Each (ky,kz)-sample was then grouped into radial spokes and ordered according to the golden-ratio [44]. Image reconstruction was performed with a combined parallel-imaging compressed sensing algorithm, with soft-gating of the source data based on intrinsic respiration motion navigation [42, 43, 45]. Temporal and spatial sparsity were enforced. Post-processing software corrected for phase errors by estimating the background phase from a polynomial fit on the phase of the static tissue.

Subjects were analyzed in a blinded fashion by KGJ, a pediatrics resident with 2 years research experience in CMR post-processing. For training, a series of 10 subjects were blindly measured with feedback from a cardiologist with > 8 years clinical experience in CMR (SAM). The data from these feedback sessions were not included in the study. Subjects were placed in a random order by a random number generator for analysis such that the effect of training would be minimized. A random number generator was used to select 20 subjects for which measurements were repeated with a minimum of 2 weeks between analyses to obtain intra- and inter-rater reliability (KGJ, SAM).

2D planimetric images were analyzed on QMassMR (MEDIS; Leiden, the Netherlands). A parallel stack of short-axis images were manually traced for endocardial borders to calculate a volume for each image plane, which were then summed to produce end-diastolic volumes (EDV) and end-systolic volumes (ESV). Both the 2- and 4-chamber views were used to cross-reference for ventricular planimetry [16]. A slice was considered to contain primarily atrium when < 50% of the blood volume was encircled with myocardium, outflow tracts were included in ventricular volumes up to the semilunar valve cusps, and trabeculations and papillary muscles were considered part of the blood pool [14]. Stroke volume (SV) was calculated as EDV-ESV for each respective ventricle. 2D PC measurements of AVFF were analyzed on QFlowMR (MEDIS). A region of interest was traced around the AV valve with the velocity therein integrated to determine the flow volume [15]. A sub-group analysis was performed between 2D PC AVFF and 4DF static AVFF. However, for homogeneity across the study cohort, all calculations of AIM_stat utilized the 4DF CMR measurement of AVFF.

4DF analyses were performed using cloud-based software (Arterys, San Francisco, California, USA). Measurements of aortic and pulmonary artery flow were made at the level of the sinotubular junction for the aorta and midway between the pulmonary valve and pulmonary artery bifurcation, respectively, as per published recommendations [14, 28]. With reference to the 2- and 4-chamber cross-sections, an initial plane was created at end-systole at the mid-leaflet level of the AV valve and orthogonal to flow [16, 27]. Flow was visualized by velocity vectors depicting blood flow directionality in real-time and color-coded for magnitude. Flow through the AV static plane was semi-automatically traced by the software during diastole and manually corrected, as needed. The retrospective valve-tracking AV plane was manually adjusted through diastole to remain near the mid-leaflet level and orthogonal to flow [37], with similar semi-automated tracing of the AV inflow as with the static plane. The regurgitant jet was measured by a plane perpendicular to the jet immediately distal to regions of aliasing, if present [24, 40]. In cases of multiple jets, separate planes were created for each jet and the volume of each jet summed. Care was taken to ensure tracings for each individual jet did not overlap. Time necessary to complete post-processing was recorded for both AIMs and JET.

Our study compares four methodologies calculating regurgitant volume (RVol). Given the clinical utility of regurgitant fraction (RF), we report RF as our outcome. RF was calculated as the quotient of AVVR and AV inflow by the formula RVol/[RVol + semilunar valve net flow (SLNF)]× 100 based on conservation of mass dictating that net inlet and outlet flows are equal. RF was classified as mild (< 30%), moderate (30–39%), moderate-severe (40–49%), or severe (> 50%) [13]. The four methods of calculating RVol were as follows (Figs. 1 and 2; Supplementary videos):

A power analysis demonstrated that for a paired t-test, 44 subjects would be necessary with alpha of 0.05 and beta of 10% (power 90%) to detect a 10% difference in regurgitant volume between subjects, allowing for 20% variance (effect size: 0.5).

Continuous data were described as mean (±standard deviation (SD)) and median (interquartile range (IQR)) as appropriate based on normality assessment with Shapiro-Wilk tests and histograms. Correlation between modalities was determined via Pearson’s (r) or Spearman’s (ρ) correlation coefficients, as appropriate. Variance between measurements was described as median difference (IQR) and absolute median difference (IQR) or mean difference (±SD) and absolute mean difference (±SD), as appropriate. The presence of any systematic difference between measurements was determined via the Wilcoxon signed rank test or Student’s paired t-test. All differences were calculated in the order presented, e.g. VOL vs. AIM calculated as VOL - AIM. Bland-Altman plots were used to visualize and compare agreement between methods [46, 47]. Bland-Altman plots show variance as limits of agreement (LOA), calculated as mean ± SDx1.96. However, this does not appropriately represent our data given the skewed, non-normal distribution, thus consideration should be given to the median difference (IQR) provided in the table as additional representation of the distribution of data. Linear regression scatter plots were used to further visualize distribution and correlation of data. Spearman’s correlation coefficients are presented on the linear regression analyses as the non-parametric complement.

Reliability was assessed with intra- and inter-rater repeated measurements (KGJ, SAM) [48]. Intra-class correlation coefficients (ICC) were calculated using a two-way mixed model with absolute agreement, reporting the single measures ICC with 95% confidence interval (CI) [49]. Statistical significance was considered if a two tailed p-value was < 0.05. All statistical analyses were performed using SPSS (v.25, Statistical Package for the Social Sciences, International Business Machines, Inc., Armonk, New York, USA).

Measurement of RF by JET was similar to VOL (median difference (IQR) -1.5% (− 8.3–7.2%), p = 0.62) with moderate-strong correlation (ρ = 0.73, p < 0.001) (Table 3). While there were moderate-strong correlations between VOL and both AIM methods (ρ = 0.69–0.70, p < 0.001), the AIM methods using static (AIM_stat) and valve-tracking planes (AIM_track) were significantly different from VOL and underestimated RF with median differences of 4.3% [− 2.0–15.7%] (p = 0.027) and 6.9% [0.2–15.0%] (p = 0.006), respectively. These methods similarly underestimated RF relative to JET, though to a slightly lesser degree, with median differences of 3.6% [− 12.0–1.8%] (p = 0.005) and 5.8% [− 13.7–0.3%] (p < 0.001), respectively. AIM_stat and AIM_track were highly correlated (ρ = 0.98, p < 0.001). RF by AIM_stat was slightly larger than AIM_track by a median difference (IQR) of 1.7% [0.1–4.3%] (p < 0.001).

In assessing internal consistency, BA and scatter plots comparing all four methods demonstrated narrower limits of agreements and more precise spread of data for the JET method compared to VOL and AIM methods than the VOL method compared to JET and AIM methods (Figs. 3 and 4).

For the 13 subjects in whom 2D PC AVFF data was available, a sub-group analysis was performed. AVFF measurements were not significantly different between 2D PC and 4DF CMR (mean difference 0.4 ± 9.5 mL/beat, p = 0.892) and were highly correlated (Pearson’s r = 0.962, p < 0.001). AIM_stat using 2D PC AVFF calculated a lower RF than AIM_stat using 4DF static AVFF though this difference was non-significant (median difference (IQR) 2.8% (− 8.1–5.5), p = 0.701)). As a result, in this smaller cohort, 2D AIM_stat correlated worse with VOL (ρ =0.736, p = 0.004) than 4DF AIM_stat with VOL (ρ =0.824, p = 0.001). AIM_stat by both modalities underestimated RF relative to VOL by median differences of 9.1–11.7% (p < 0.05).

Intra- and inter-observer reliability was tested on 20 randomly selected subjects (45%): 12 had tricuspid regurgitation and 8 had mitral regurgitation. Intra-observer correlations for the four methods were excellent (ICC 0.97–0.98) with no significant differences between measurements (p = 0.88–0.97) (Table 4). Mean differences were all close to zero with narrow LOAs (Fig. 5). Inter-observer correlations for the 4 methods were also excellent (ICC 0.94–0.96). Measurements of RF were not significantly different between raters using the AIM and JET methods (p = 0.22–0.47), with small mean differences and narrow LOAs. However, inter-observer measurements of VOL were significantly different (p = 0.046), although the clinical significance of this difference was small (2.41% ± 5.05) and the variance of this difference was smaller than inter-observer differences for JET and AIM methods (Fig. 5). Intra- and inter-observer analysis of planimetry measurements demonstrated excellent reliability of left ventricular (LV) and right ventricular (RV) volumes (ICC = 0.99) with small, clinically insignificant, median differences (Table 4).

A sub-group analysis found that agreement between VOL and JET measurements was not affected by laterality of AVVR (Table 5). However, JET had better precision with VOL and AIM methods for assessment of tricuspid RF (ρ = 0.66–0.85, p < 0.001; lower measurement variability) than VOL with JET and AIM methods (ρ = 0.58–0.66, p < 0.001). Conversely, the JET method was less precise with VOL and AIM for assessment of mitral RF (ρ = 0.66–0.69, p < 0.001; higher measurement variability) than VOL with JET and AIM (ρ = 0.66–0.79, p < 0.001) (Table 5). Measurements of intra- and inter-observer reliability were similar for mitral and tricuspid AVVR.

This is the first study demonstrating validity of the JET method to accurately and reliably measure AVVR compared to multiple indirect methods in a diverse pediatric cohort with CHD. Our findings support the results of prior work on the JET method in other patient populations. A retrospective study of 21 adults without CHD found that RF directly measured by 4DF CMR was highly reliable and accurate compared to the VOL method [40]. The correlation and measurement variability noted between the JET and VOL methods in our cohort are similar to prior data from pediatric and adult cohorts [24, 29].

Although this is the first such report in children, the JET method by 4DF CMR has been previously found in adults to have high accuracy and reliability relative to the VOL method, particularly in measuring tricuspid regurgitation [40]. 4DF CMR post-processing allows for visualization of flow to aid in the positioning of measurement planes through the use of velocity vectors [37]. Prior work has demonstrated improved accuracy for the AIM method when the atrioventricular plane is placed at the peak inflow velocity as opposed to the valvular plane [37, 50]. The importance of a measurement plane perpendicular to flow likely extends to JET planes and may explain the relatively worse correlation between measurements of mitral RF between the JET and the VOL and AIM methods. Tricuspid regurgitant jets tend to be more central and laminar in orientation as opposed to mitral jets which are more commonly eccentric with a large angle change during systole [40]. In a cohort of 32 children and adults with repaired atrioventricular septal defect (AVSD) and left-sided regurgitation, the regurgitant jets were found to be non-circular, dynamic, and eccentric with a median angulation change of 30–36° during systole [24]. These features increase the technical difficulty of maintaining an orthogonal plane to the mitral regurgitant jet through systole. As can be seen in a subject with repaired AVSD and resultant mitral cleft (Fig. 2b), the large eccentric regurgitant jet which projects along the inferior mural aspect of the left atrium throughout systole complicates creating a true cross-sectional plane. Five of the 18 (28%) subjects in our cohort with mitral regurgitation had a history of AVSD which may have affected accuracy of our JET measurements despite excellent reliability. Our data highlight the clinical applicability of the JET method in pediatric CHD requiring evaluation of tricuspid regurgitation. Although the results were also promising in cases of mitral regurgitation, considerably eccentric or dynamic jets may affect the accuracy of the JET method.

Our initial intent was to use VOL as the clinical standard to which to compare JET and AIM methods. Upon analysis of the data we found that all four methods were fairly consistent, but that VOL was not the most precise measurement -- which raises the question of its role as the clinical standard. The accuracy of VOL is affected by measurement errors inherent in planimetry, particularly given the practice variation in basal slice inclusion and contour selection, which may have contributed to lower measurement precision [16, 51]. However, our planimetry reliability analyses found excellent intra- and inter-observer reliability for both LV and RV measurements. Similarly, our results demonstrated excellent reliability for the VOL method, supporting the results from multiple prior works [9, 17, 29, 40]. Two additional practical concerns with the VOL method arose from our data. In our cohort, VOL classified more subjects with moderate or severe regurgitation than the JET and AIM methods. This may result in greater consideration for surgery when VOL is used to measure regurgitation. Furthermore, a limitation to the use of VOL as the clinical standard are cases in which the indirect calculation results in a negative RF (Table 2). Negative RFs were found in 5/44 (11%) subjects by the VOL method as well as in a separate 6/44 (14%) and 7/44 (16%) by AIM_stat and AIM _track, respectively.

The AIM methods systematically underestimated RF compared to VOL and JET methods. Though these differences may be clinically insignificant in certain patients, for subjects with mild regurgitation (~ 60% of our cohort), these differences represented up to 25% of their total RF. AIM_stat measured slightly but consistently larger than AIM_track, consistent with prior studies demonstrating that static AV planes overestimate RVol and RF relative to tracking planes [23, 37, 52]. The differences found in our study were smaller than prior reports and may be due to our methodology; determination of a “static plane” by 4DF CMR has significantly more flexibility than a standard 2D CMR scan and may have increased accuracy of the measurements. This is supported by our sub-group analysis demonstrating that AIM_stat calculated using 2D PC AVFF measurements correlated worse with and had a larger median difference from VOL than AIM_stat calculated using 4DF CMR static AVFF. Prior work using 4DF CMR and retrospective valve tracking annular inflow methods do not report their calculated AVVR as outcomes, instead comparing net inlet and outlet flows for internal validation, which limits direct comparison to our results as well as general clinical applicability [22, 25]. Recent work has demonstrated feasibility of an automatic valve-tracking 4DF CMR technique with improved reliability and internal consistency in measuring net valve flow across all four cardiac valves compared to manual valve-tracking technique in a large cohort of subjects with CHD [36]. However, their RF measurements were not compared to a clinical standard (such as VOL) or outcome measures. Future work assessing the accuracy of the automatic valve-tracking AIM method may demonstrate improved accuracy than found in our study with manual valve-tracking.

2D CMR requires technical expertise from technologists and physicians to prescribe AV planes during the acquisition in patients with complex CHD anatomy, resulting in scan durations of 1–1.5 h [41, 53]. This has important implications on anesthetic requirements in pediatrics and on clinical efficiency. 4DF CMR, as a single acquisition with all planes created during post-processing, takes ~ 10 min after an initial localizer sequence. Acquisition of standard 2D PC requires multiple localizing sequences to achieve optimal orthogonal planes. Children are more likely to move in-between sequences, requiring repeat localizer sequences and longer exam times. This has resulted in the adoption of 4DF CMR for acquisition of all flow measurements at our institution, though 2D cine acquisitions are still necessary for planimetry.

Our post-processing analysis times for the AIM and JET methods were short and represent an opportunity to improve clinical flow and resource utilization in the MRI workflow. Our analysis time for the AIM_track method was significantly shorter than prior reports of 15–35 min per patient, likely due to the use of semi-automated software that adjusted to through plane motion [23, 25]. The JET method analysis time was relatively short (3.1 ± 2.1 min) irrespective of number and morphology of regurgitant jets.

Several limitations were present in our study. VOL in this study utilized 2D planimetry measurements and 4DF PC semilunar flow measurements, which precludes comparison of a true 2D VOL measurement to the 4DF JET. However, 4DF PC measurements of semilunar flow have been previously validated with excellent internal consistency and accuracy compared to 2D PC CMR [22, 30, 31, 35, 39], thus we do not believe this negatively impacted the validity of our results. We excluded subjects with significant semilunar regurgitation and our data may not be generalizable to those populations. Arterial encoding speeds were utilized in our cohort as part of our institution’s clinical practice based off prior published findings demonstrating reliable quantification of venous blood flow at arterial VENCs with the use of IV contrast, although this may have contributed to underestimation of the AIM method [41, 54]. It is not our clinical practice to perform multiple 4DF sequences at different VENC settings, though this may allow for more accurate data. Future application of phase unwrapping software may allow for a single, low-VENC 4DF sequence. Despite arterial VENCs, aliasing of regurgitant jets was common. However, tracing the jets distal to the aliasing segment allowed for accurate and reproducible data. We employ contrast agents for all our 4DF acquisitions. Recent research has found dose-dependent retention of gadolinium-based contrast in the brains of patients despite normal renal function, though long-term effects from chronic exposure remain unknown [55]. Although ferumoxytol is used clinically at an increasing number of centers for cardiac imaging, there remains a black box warning in pediatric imaging due to the small risk of adverse reactions, most notably hypotension [56]. Further study is necessary to clarify the risk:benefit ratio of contrast administration for these studies. Although our methodology for calculating AVVR differed slightly from published recommendations we accounted for even a small amount of semilunar regurgitation in order to optimize accuracy when comparing methodologies [13, 16, 53, 57]. Lastly, this study did not validate direct AVVR quantification against clinical outcome measures, as this was beyond the scope of the study.