Determining the dosage of Vitamin K antagonists (VKA) is a complex and continuous process, influenced by many different factors. Dosage is currently determined by our anticoagulation doctors and consultants, based on decision-making support tools in the dosage programme. In recent years, we have been investigating whether an algorithm can help to tailor the dosage even more precisely to an individual patient.
In 2018, Star-shl partnered with Pacmed, an organisation specialising in machine learning and developing AI solutions for the healthcare sector. Pacmed worked closely with Star-shl to design a self-learning algorithm that uses artificial intelligence to more accurately predict expected INR values based on a patient’s medical history, medication, previous INRs and dosing regimens over the past five years. The algorithm is capable of analysing millions of variables. Based on these variables, the algorithm can predict an appropriate dose, leading to a projected INR for the patient.
In 2019, we demonstrated that the algorithm could help determine dosage recommendations in stable patients. INR prediction was not always possible for patients from the categories 'Experimenteel' (dosage recommendation has been determined but should always be double checked) and 'Niet' (the algorithm could not determine a dosage recommendation). This is because there are a number of variables that the algorithm cannot comprehend, making it more difficult to predict the INR. Examples of such variables include the patient’s regimen compliance, diet and possible interaction with other medications. These factors create a ‘blind spot’ in the algorithm.
In 2020, based on these findings, the algorithm was modified with the help of Star-shl’s medical expertise. Instead of predicting the next INR and the corresponding dosage, the algorithm was adapted to provide decision-making support for the medical professionals determining the dosage (anticoagulation physicians and consultants). The adjustment was based on data from the 10% ‘best’ dosage consultants. Using their data, the algorithm was trained to give a good dosage recommendation. This recommendation can be used to support decision-making for dosing Vitamin K antagonists in our patients.
We have recently tested the modified algorithm under the supervision of Chantal Visser, PhD researcher with the Department of Hematology at Erasmus MC. To that end, we compared the recommendations provided by the Pacmed algorithm to the recommendations resulting from the decision-making tools that are already available and to the human recommendations provided by our dosage consultants. In 280 patients, we looked at the actual dosage recommendations in 2018, and the recommendations based on the various existing decision-making tools: Pacmed, Trodis (the programme currently used by Star-shl), and the Beinema II algorithm, which was defined in 2016 and is currently used in several other anticoagulation clinics in the Netherlands. We then asked the dosage consultants to evaluate the resulting recommendations.
Six Star-shl dosage consultants took part in this study. For each patient, they received a recommendation from one of the decision-making tools or the actual dosage, in order to determine dosage based on that recommendation. Each patient was presented to two dosage consultants. There were four recommendations: the dosage at that time, the Trodis recommendation, the Beinema II recommendation and the Pacmed recommendation. The dosage consultants did not know which decision-making tool had been used for each recommendation. In addition, we looked at how closely the dosages determined by the various dosage consultants corresponded. Finally, we asked the dosage consultants to assign a score from 1 to 10 for the recommendation they used.
The initial analyses show that there were small but significant differences between the median dosages after determining dosage based on the different decision-making tools. For example, the dosage recommendation obtained with the help of the Pacmed algorithm was lower than the actual recommendation given in 2018. Conversely, if a dosage consultant used Trodis or Beinema II, the dosage recommendation was higher than the actual dosage determined in 2018. If we look at the different patient groups, Pacmed’s recommendations were changed more often. Dosage consultants were less likely to change Beinema II’s recommendations, also in the categories 'Experimenteel' and 'Niet'. Beyond that, it became apparent that Star-shl dosage consultants gave similar dosage recommendations. There were no major differences between employees. Finally, looking at the score given by the participants, Beinema II achieved the highest ranking, followed by Trodis and Pacmed.
Based on these results, a reference session was held with all those involved to see what we could conclude from these results. In any case, we can conclude that developing an algorithm based on AI (machine learning) to predict the next INR appears to be more difficult than initially expected. Following from the reference session, we will sit down with Pacmed to consider possible further analysis to reach definitive conclusions.