In patients undergoing echocardiographic evaluation of cardiac function, preliminary assessment by artificial intelligence (AI) is superior to initial sonographer assessment, in line with late breaking research presented in a Hot Line session today at ESC Congress 2022.
There was much excitement in regards to the use of AI in medicine, however the technologies are rarely assessed in prospective clinical trials. We previously developed one in every of the primary AI technologies to evaluate cardiac function (left ventricular ejection fraction; LVEF) in echocardiograms and on this blinded, randomised trial, we compared it face to face with sonographer tracings. This trial was powered to indicate non-inferiority of the AI in comparison with sonographer tracings, and so we were pleasantly surprised when the outcomes actually showed superiority with respect to the pre-specified outcomes.”
Dr. David Ouyang of the Smidt Heart Institute at Cedars-Sinai, Los Angeles, US
Accurate assessment of LVEF is important for diagnosing heart problems and making treatment decisions. Human assessment is usually based on a small variety of cardiac cycles that may end up in high inter-observer variability. EchoNet-Dynamic is a deep learning algorithm that was trained on echocardiogram videos to evaluate cardiac function and was previously shown to evaluate LVEF with a mean absolute error of 4.1-6.0% . The algorithm uses information across multiple cardiac cycles to minimise error and produce consistent results.
EchoNet-RCT tested whether AI or sonographer assessment of LVEF is more continuously adjusted by a reviewing cardiologist. The usual clinical workflow for determining LVEF by echocardiography is that a sonographer scans the patient; the sonographer provides an initial assessment of LVEF; after which a cardiologist reviews the assessment to offer a final report of LVEF. On this clinical trial, the sonographer’s scan was randomly allocated 1:1 to AI initial assessment or sonographer initial assessment, after which blinded cardiologists reviewed the assessment and provided a final report of LVEF (see figure).
The researchers compared how much cardiologists modified the initial assessment by AI to how much they modified the initial assessment by sonographer. The first endpoint was the frequency of a greater than 5% change in LVEF between the initial assessment (AI or sonographer) and the ultimate cardiologist report. The trial was designed to check for noninferiority, with a secondary objective of testing for superiority.
The study included 3,495 transthoracic echocardiograms performed on adults for any clinical indication. The proportion of studies substantially modified was 16.8% within the AI group and 27.2% within the sonographer group (difference -10.4%, 95% confidence interval [CI] -13.2% to -7.7%, p<0.001 for noninferiority, p<0.001 for superiority). The security endpoint was the difference between the ultimate cardiologist report and a historical cardiologist report. The mean absolute difference was 6.29% within the AI group and seven.23% within the sonographer group (difference -0.96%, 95% CI -1.34% to -0.54%, p<0.001 for superiority).
Dr. Ouyang said: “We learned so much from running a randomised trial of an AI algorithm, which hasn’t been done before in cardiology. First, we learned that this sort of trial is very feasible in the fitting setting, where the AI algorithm could be integrated into the standard clinical workflow in a blinded fashion. Second, we learned that blinding really can work well in this case. We asked our cardiologist over-readers to guess in the event that they thought the tracing they’d just reviewed was performed by AI or by a sonographer, and it seems that they couldn’t tell the difference – which each speaks to the strong performance of the AI algorithm in addition to the seamless integration into clinical software. We consider these are all good signs for future trial research in the sphere.”
He concluded: “We’re excited by the implications of the trial. What this implies for the long run is that certain AI algorithms, if developed and integrated in the fitting way, may very well be very effective at not only improving the standard of echo reading output but additionally increasing efficiencies in effort and time spent by sonographers and cardiologists by simplifying otherwise tedious but necessary tasks. Embedding AI into clinical workflows could potentially provide more precise and consistent evaluations, thereby enabling earlier detection of clinical deterioration or response to treatment.”
Source:
European Society of Cardiology (ESC)